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Preface 


The mathematician John von Neumann once urged the information theorist 
Claude Shannon to assign the name entropy to the measure of uncertainty 
Shannon had been investigating. After all, a structurally identical measure 
with the name entropy had long been an element of statistical mechanics. 
Furthermore, “No one really knows what entropy really is, so in a debate you 
will always have the advantage.” Most of us love clever one-liners and allow 
each other to bend the truth in making them. But von Neumann was wrong 
about entropy. Many people have understood the concept of entropy since it 
was first discovered 150 years ago. 

Actually, scientists have no choice but to understand entropy because the 
concept describes an important aspect of reality. We know how to calculate and 
how to measure the entropy of a physical system. We know how to use entropy 
to solve problems and to place limits on processes. We understand the role of 
entropy in thermodynamics and in statistical mechanics. We also understand 
the parallelism between the entropy of physics and chemistry and the entropy 
of information theory. 

But von Neumann’s witticism contains a kernel of truth: entropy is difficult, 
if not impossible, to visualize. Consider that we are able to invest the concept 
of the energy of a rod of iron with meaning by imagining the rod broken into its 
smallest parts, atoms of iron, and comparing the energy of an iron atom to that 
of a macroscopic, massive object attached to a network of springs that model 
the interactions of the atom with its nearest neighbors. The object’s energy is 
then the sum of its kinetic and potential energies - types of energy that can be 
studied in elementary physics laboratories. Finally, the energy of the entire 
system is the sum of the energy of its parts. 

These imaginative transitions - first to analyze a whole into its parts, sec- 
ond to compare each part with a familiar object, third to recognize the quan- 
tity sought in the familiar object, and finally to recompose the whole out 
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of its parts - fail to shed light on entropy. The difficulty lies in recognizing 
the entropy in the smallest part. A single, localized molecule or atom has no 
entropy. Nevertheless, a rod of iron has entropy. Entropy is not a localized 
phenomenon at which we can point, even in our imaginations, and say: “Look! 
There is entropy.” And, if we insist on trying to understand a subject in ways 
inconsistent with its nature, we will be disappointed. 

This student guide introduces us to the ways in which entropy can be under- 
stood. It emphasizes conceptual foundations and exemplary illustrations and 
distinguishes among different kinds of entropy: thermodynamic entropy, the 
entropy of classical and quantized statistical systems, and the entropy of infor- 
mation. These entropies differ in the classes of systems to which they apply. 
But all name the same concept in the sense that they reduce to each other when 
they can be applied to the same object. 

Several features make this text appropriate as a student guide and appro- 
priate for self-study. Never does the text depend upon “It can be shown.” 
Derivations are included and each step of each derivation is made explicit. 
Mathematical techniques are used and reused and each piece of physics is 
revisited in different contexts. Worked examples illustrate important applica- 
tions. Italicized words usually mean, “Note well. Here is a definition.” These 
definitions, often in expanded form, are collected in a Glossary. The answers to 
all end-of-chapter problems are appended and an Annotated Further Reading 
list identifies sources and suggests books for further study. Rudolph Clausius, 
Ludwig Boltzmann, Max Planck, Claude Shannon, and others developed the 
concept of entropy over a period of 100 years. I have not been reluctant to 
assign credit where credit is due and to associate people and developments 
with dates. A Time Line organizes this information. A Formulary contains use- 
ful identities and expansions. 

The middle chapters of this guide, Chapters 2 to 7, compose an entropy- 
centered introduction to equilibrium statistical mechanics. Not only is entropy 
a smooth pathway to major results, including blackbody radiation and the ideal 
quantum gases, but concentrating on entropy keeps one close to the physics. 

The experienced user of thermal and statistical physics texts will notice sev- 
eral original features. One of these is the way statistical entropy is constructed, 
step-by-step, out of desired properties. This first phase of this construction is 
accomplished in Section 2.4 and the last in Section 4.5. Another is that the 
entropy of a classical system is derived from purely classical presuppositions 
in Section 2.4. Most texts do not bother with constructing a coherent classical 
statistical mechanics, but instead make do with the semi-classical limit of the 
corresponding quantum description. A third is that the ideal Fermi gas (Section 
6.3) and ideal Bose gas (Sections 7.3 and 7.4) are developed from the properties 
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of their entropy functions via the average energy approximation (Section 6.4) 
rather than from partition functions and density of states functions. The logic 
of these constructions and derivations is uncluttered and efficient. 

What has been most important to me in writing this guide has been to 
closely correlate verbal and mathematical formulations and to translate sim- 
ple concepts into simple mathematics. If the whole is complex, I have always 
meant the parts to be simple. My goal has been to write simply of the simple 
and clearly of the complex. 

If I have, in some measure, been successful, no small credit is due those 
who helped me. Ralph Baierlein, in particular, deserves my thanks for thought- 
fully reading and rereading the entire text and giving unselfishly of his expert 
advice. Ralph saved me from several consequential missteps and improved 
the readability of the text. Ralph also summarized content from German lan- 
guage sources. Anthony Gythiel provided translations from the German. Rick 
Shanahan also read and commented thoughtfully on the entire text. Dale Allison, 
Clayton Gearhart, Galen Gisler, Bob Harrington, Carl Helrich, and Bill Peter 
read and commented on various parts of the text. Hans von Baeyer, Andrew 
Rex, and Wolfgang Reiter graciously answered my questions. My students in a 
thermal physics course at Wichita State University constantly reminded me of 
my purpose. I am indebted to these students, colleagues, and friends. 
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1.1 Thermodynamics and entropy 

The existence of entropy follows inevitably from the first and second laws 
of thermodynamics. However, our purpose is not to reproduce this deduction, 
but rather to focus on the concept of entropy, its meaning and its applications. 
Entropy is a central concept for many reasons, but its chief function in thermo- 
dynamics is to quantify the irreversibility of a thermodynamic process. Each 
term in this phrase deserves elaboration. Here we define thermodynamics and 
process', in subsequent sections we take up irreversibility. We will also learn 
how entropy or, more precisely, differences in entropy tell us which processes 
of an isolated system are possible and which are not. 

Thermodynamics is the science of macroscopic objects composed of many 
parts. The very size and complexity of thermodynamic systems allow us to 
describe them simply in terms of a mere handful of equilibrium or thermody- 
namic variables, for instance, pressure, volume, temperature, mass or mole num- 
ber, internal energy, and, of course, entropy. Some of these variables are related 
to others via equations of state in ways that differently characterize different 
kinds of systems, whether gas, liquid, solid, or composed of magnetized parts. 

A thermodynamic system undergoes a thermodynamic process when the vari- 
ables that describe the system change from one set of values to another, that is, 
change from one thermodynamic state to another. Thus, we describe a thermody- 
namic process by specifying an initial state, a final state, and specific conditions 
under which the change occurs. Specific conditions include: iso-energetically, that 
is, with constant energy as would occur when the system is completely isolated; 
adiabatically, that is, within a thermally insulating boundary; and quasistati- 
cally, that is, so slowly the system occupies a continuous sequence of thermody- 
namic states each one of which is characterized by thermodynamic variables. 
Thermodynamics per se says nothing about the rate at which a process unfolds. 
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The laws of thermodynamics also limit the ways in which a thermodynamic 
system may proceed from one state to another. The first law of thermodynamics 
is the law of conservation of energy applied to the internal energy of a ther- 
modynamic system. The internal energy excludes the energy of the position or 
motion of the system as a whole. According to the first law there are only two 
ways to change the internal energy of a given thermodynamic system: (1) heat- 
ing or cooling the system and (2) working on the system or arranging for the 
system to work on its surroundings. Quantitatively the first law is 

AE=Q + W (1.1) 

where A E j^= E f - E i J is the increment in the internal energy £ of a thermody- 
namic system as it transitions from an initial state with energy £ to a final state 
with energy E f . The quantity Q is the energy absorbed by the system when 
heated, and W is the work done on the system during the transition. These are 
signed quantities so that when Q < 0, the system is cooled rather than heated, 
and when W < 0, work is done by rather than on the system. (Note that some 
texts use the symbol W in the opposite way so that when W < 0, work is done 
on rather than by the system. In this case the first law of thermodynamics is 
AE = Q-W instead of (1.1).) 

When these changes are indefinitely small the first law becomes 

dE = SQ + 5W. (1.2) 

Here the different notations, dE versus 5Q and 5W, emphasize that while E is 
a variable that describes the state of a system, Q and W are not. The heat quan- 
tities, Q and 5Q, and their work correlatives, W and 5W, merely indicate the 
amount of energy transferred to or from a system in these ways. (We have no 
standard way of distinguishing between a differential of a state variable and an 
equivalently small infinitesimal that is not the differential of a state variable. 
Here I use d to signal the first as in dE and <5 to signal the second and as in 5Q 
and 5W. Other texts adopt other solutions to this notational problem.) Thus, a 
thermodynamic system contains energy but does not, indeed cannot, contain 
heat or work, as suggested in Figure 1.1. 


1.2 Reversible and irreversible processes 

All interactions among fundamental particles are reversible. The simplest 
interaction is a collision between two fundamental particles. One particle 
approaches another; they interact via gravitational, electromagnetic, or nuclear 
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Figure 1 . 1 The internal energy £ of a thermodynamic system can be changed in 
only two ways: (1) heating or cooling and (2) working. 

forces; then they recede from each other. Interestingly, a film of such a colli- 
sion would make as much physical sense proceeding backwards as proceeding 
forwards. It may help us to think of a white cue ball striking a colored billiard 
ball, the latter initially at rest. While some care may be required to set both 
balls in motion with reversed velocities in such a way that the colored billiard 
ball strikes the cue ball and both recover their initial states of rest and motion, it 
can be done. And certainly this reversal can be imagined without supposing the 
violation of any of Newton’s laws of motion. For there is nothing in the laws of 
motion or in the way fundamental particles interact that prefers one direction 
of time over the other. We say that interactions among fundamental particles 
are time reversible. 

The same cannot be said of the processes of a thermodynamic system com- 
posed, as it is, of many parts. Thermodynamic processes are, in fact, typically 
irreversible. A cup of hot coffee sitting on the kitchen table always cools down. 
We never observe a cup of initially room-temperature coffee extracting energy 
from the air and heating up. When a hot object and a cold one are in thermal 
contact, the hot object always cools and the cold object always heats until both 
reach an intermediate temperature. We never observe these processes occur- 
ring in reverse order. A backwards-running video of a thermodynamically irre- 
versible process appears implausible. 
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There is, however, a special sense in which a thermodynamic process can mean- 
ingfully be reversed. If making only an indefinitely small change in the system 
or its environment reverses the direction of the process, the process is thermody- 
namically reversible. Sadi Carnot (1796-1832) invented the concept of thermo- 
dynamic reversibility in order to articulate and prove what is now called Carnot’s 
theorem: The most efficient heat engine is one that operates reversibly. 
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(a) (b) 

Figure 1.2 (a) A frictionless piston reversibly compresses a fluid, (b) Two systems 
with an infinitesimal temperature difference heat and cool each other reversibly. 


A reversible thermodynamic process must be (1) quasistatic, that is, indefi- 
nitely slow, and (2) without friction or dissipation (internal friction). Consider, 
for instance, the piston compressing the fluid illustrated in Figure 1.2a. The 
piston quasistatically and frictionlessly compresses the fluid when the force the 
piston exerts on the fluid, PA + £, is an indefinitely small amount e larger than 
the force PA the fluid of pressure P exerts on the piston head of area A. When, 
instead, the force exerted by the piston on the fluid is infinitesimally less than 
PA, that is, PA - £, the fluid expands quasistatically. Thus, these are thermo- 
dynamically reversible changes because their direction can be reversed by an 
infinitesimal change in either the system or its environment. In similar fashion, 
the heating or cooling of one system by another is reversible when an infinites- 
imal temperature difference is maintained between the two as shown in Figure 
1.2b. Clearly, thermodynamic reversibility is an ideal that can be approached 
but never fully realized. All actual thermodynamic processes are irreversible. 

To summarize: all fundamental processes are reversible because one can 
imagine reversing their direction of change without violating a law of physics. 
On the other hand, all non-idealized thermodynamic processes are irreversible. 
Thermodynamic processes, in practice, always proceed in one direction. 

Loschmidt’s paradox 

Yet if, as is widely assumed, a thermodynamic system is composed of many 
fundamental particles and a thermodynamic process is composed of many 
fundamental interactions, why are not all thermodynamic processes revers- 
ible? Johann Loschmidt (1821-1895) asked this question in 1876. We still 
have no fully satisfactory answer. That many reversible fundamental pro- 
cesses do not necessarily compose a reversible thermodynamic process is 
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known as Loschmidt’s paradox or the reversibility paradox. Our failure to 
resolve Loschmidt’s paradox suggests that the laws governing the interactions 
of fundamental particles do not form a complete picture of nature and need 
to be supplemented with additional physics equivalent to the second law of 
thermodynamics. 


Example 1.1 Reversible or irreversible? 

Problem: A piston quasistatically compresses a gas enclosed in a chamber. In 
order to compress the gas the piston must overcome a force of 0.01 N caused 
by the piston head rubbing against the sides of the piston chamber. Is the work 
performed on the gas reversible or irreversible? 

Solution: This work is performed quasistatically but not without friction. 
Therefore, the work is irreversible. Alternatively, in order to change the quasi- 
static compression into a quasistatic expansion the force exerted on the piston 
must change by the finite amount of 2 x 0.01 N. 


1.3 The second law of thermodynamics 

There are a number of logically equivalent versions of the second law. Perhaps 
not surprisingly, the earliest versions are the most readily grasped. Rudolph 
Clausius (1822-1888) was the first, in 1850, to identify a version of the second 
law of thermodynamics. For this purpose Clausius elevated everyday observa- 
tions about heating and cooling to the status of a general law: A cooler object 
never heats a hotter object. 

By heating (cooling) an object we mean transferring energy to (from) the 
object without doing work. And by cooler object we mean one with lower tem- 
perature and by hotter object we mean one with higher temperature. We will 
discuss temperature in more detail later. For now it suffices to consider temper- 
ature as the degree of hotness measured on some scale. 

Heat reservoirs 

Clausius’s and other versions of the second law of thermodynamics are most 
conveniently expressed in terms of heat reservoirs. By definition, a heat reser- 
voir maintains the same temperature regardless of how much heating or cool- 
ing it experiences. Thus a heat reservoir has infinite heat capacity just as if it 
were indefinitely large. In the language of heat reservoirs Clausius’s second 
law is: A process whose only result is to cause one heat reservoir with tem- 
perature T ( . to lose energy Q through cooling and a hotter heat reservoir with 
temperature T H > T c to gain energy Q through heating never occurs. 
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(a) (b) 

Figure 1.3 (a) A process whose only result is to cause one heat reservoir with 
temperature T. to lose energy Q c through cooling and cause a hotter heat reser- 
voir with temperature /„ > T c to gain energy Q H through heating is forbidden by 
Clausius’s version of the second law, in this case Q H = Q c - (b) A heat engine whose 
only result is to cause a single heat reservoir to lose energy Q through cooling and 
perform an equal amount of work W[=Q] is forbidden by Kelvin’s version of the 
second law. 


In 1851 William Thomson (1824—1907), later known as Lord Kelvin, for- 
mulated a verbally distinct version of the second law. Kelvin’s second law 
concerns the operation of a heat engine, that is, a device that uses a temperature 
difference to produce work. According to Kelvin’s second law: A heat engine 
whose only result is to cause a single heat reservoir to lose energy Q through 
cooling and perform an equal amount of work W [= Q\ is impossible. 

Given the first law, Kelvin’s version of the second law is logically equiva- 
lent to Clausius’s. The version one favors is a matter of taste and convenience. 
If one wants a second law in the language of natural phenomena, one adopts 
Clausius’s second law. If one wants a second law in the language of technol- 
ogy, one adopts Kelvin’s. We illustrate these impossible processes in terms of 
which these versions of the second law are framed in Figure 1.3 and label them 
with a symbol 0 that proclaims, “forbidden by the second law.” 


The second law and irreversibility 

The second law of thermodynamics can also be expressed in terms of the lan- 
guage of irreversible processes. Recall that an irreversible thermodynamic 
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process cannot be completely reversed. Therefore, another way to say that a 
particular process is forbidden is to say that the forbidden process is the result 
of reversing a thermodynamically irreversible process. In these terms Clausius’s 
second law becomes: A process whose only result is to cool a hotter reservoir 
and heat a colder one cannot be completely reversed. Likewise, Kelvin’s second 
law becomes: A process whose only result is to dissipate work into a single heat 
reservoir cannot be completely reversed. Every time our coffee cools and every 
time we shuffle across the floor we experience or create necessarily irreversible 
processes. In the next two sections we focus on the first of these, a hotter reser- 
voir heating a colder one, in order to introduce the measure of irreversibility we 
call entropy and to reformulate the second law in terms of that measure. 


1.4 Entropy and irreversibility 

Imagine a thermodynamic system that can exist in many different thermodynamic 
states. We represent each of these states by a point in a multi-dimensional thermo- 
dynamic state space. The variables that define the state space will, of course, 
depend upon the nature of the system. For instance, because only two variables 
completely determine the state of a simple fluid, the state space of a simple fluid 
is a plane whose points are defined by locating values on two axes: one labeled, 
say, energy E and the other volume V. Now, choose any two points representing 
two different states in our thermodynamic state space and a path through the state 
space that connects these two points as in Figure 1.4. Each point on this path 
necessarily represents a thermodynamic state and the complete path necessarily 
represents a quasistatic process that connects the two endpoint states. 

Of course, non-quasistatic, for example, turbulent or explosively rapid, pro- 
cesses can also transform a system from one thermodynamic state to another. 
But non-quasistatic processes are not composed of a continuous sequence of 
thermodynamic states and cannot be represented by a path in thermodynamic 
state space. 

Therefore, a path in thermodynamic state space necessarily represents a 
quasistatic process. And quasistatic processes come in only two kinds: revers- 
ible processes that proceed without friction or dissipation and irreversible ones 
that proceed with friction or dissipation. Recall that a completely reversible 
process is also one for which the system and its environment can, with only 
infinitesimal adjustments, evolve along a path in state space in either direction, 
and an irreversible process is one for which the system and its environment 
would require finite adjustments to proceed along a path in either direction. A 
piston that very slowly compresses a gas while overcoming friction between 
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Figure 1.4 Thermodynamic state space, two states, denoted (£, , V / 1 ) and (£. 2 ,Kj), 
and a path connecting the two states. 


the piston head and the sides of the piston chamber is an example of an irre- 
versible, quasistatic process that can be represented by a path in state space. 
To summarize: every segment of a path in state space represents a quasistatic 
process that, in turn, represents either a reversible or an irreversible process. 

Entropy difference 

We seek a quantitative measure of the irreversibility of a quasistatic process. 
Rudolph Clausius was, in 1865, able to construct such a measure by discov- 
ering a state variable of an isolated system that monotonically changes along 
a path representing an irreversible process and remains the same along a path 
representing a reversible process. Thus, two states of an isolated system con- 
nected by a reversible path have the same value of this state variable and two 
states of an isolated system connected by an irreversible path have different 
values of this state variable. Clausius’s discovery of this state variable required 
an elaborate deduction from the first and second laws of thermodynamics. 
I will not reproduce that deduction. 

Clausius coined the word entropy for this state variable - a word parallel 
in structure, sound, and spelling to the word energy - and adopted S for its 
symbol. Clausius derived the word entropy from the Greek root tropy mean- 
ing “turn” and a prefix en meaning in. Thus entropy literally means “in turn” 
or “turn in.” It seems that Clausius sought a name for a concept that, in part, 
characterizes how physical systems turn , that is, how they change, evolve, 
or proceed. Clausius himself interpreted the word entropy metaphorically as 
transformation content. We will in time discover more suggestive metaphors 
for the concept of entropy. 
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In a thermodynamics based on the first and second laws the only purpose 
of Clausius’s state variable entropy is to provide comparisons between two 
states. For example, if the path connecting states 1 and 2 of an isolated system 
as illustrated in Figure 1 .4 represents an irreversible process, then either S 2 > S 1 
or Sj > S 2 . Alternatively, if the path illustrated represents a reversible process of 
an isolated system, then S 2 = S v 

The absolute entropy of a single state is much like the absolute energy of 
a single state. The first and second laws of thermodynamics sanction neither 
absolute energy nor absolute entropy but assign meaning only to energy and 
entropy differences. The third law of thermodynamics does, indeed, allow us 
to assign a meaning to the concept of absolute entropy. But before we explore 
the third law we determine, in the next several sections, how the entropy diffe- 
rence between two states of an isolated system quantifies the irreversibility of 
a process connecting those two states. 


1.5 Quantifying irreversibility 

The key to quantifying the irreversibility of an isolated system’s evolution is 
to require that the entropy increment AS of a composite system be additively 
distributed over its parts. Symbolically stated, 

as X X A - v , ( L3 ) 

where S f j is the final entropy of the system’s y'th part, S l ] is the initial entropy 
of the system’s yth part, and A.SV S f j - S tJ J is the entropy increment of 
the system’s yth part. In this way, and as illustrated in Figure 1.5, the entropy 
increment of a composite system AS is the sum of the entropy increments of 
its parts. 

To proceed further we apply these ideas to a particular process: a heat reser- 
voir with temperature T is heated by absorbing energy Q. If, alternatively, the 
heat reservoir is cooled, then Q < 0. Therefore, if the heat reservoir increments 
its entropy, that increment AS can be a function of only two quantities: the reser- 
voir temperature T and the heat Q where the sign of Q indicates whether energy 
is absorbed through heating [<2 > 0] or rejected through cooling [Q < 0]. [The 
noun heat here refers to the energy absorbed or rejected by heating or cooling.] 
The equation 


A S = f(T,Q) 


(1.4) 
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AS = AS 1 + AS 2 + AS 3 + AS 4 + AS 5 


AS, 

A S 2 

as 3 

as 4 

A S 5 


Figure 1.5 The entropy increment AS is additive over the parts of a composite 
system. 


states this relation. Note that we have imposed no special restrictions on the 
system or on this heating or cooling process. In particular the heating or cool- 
ing described in Eq. (1.4) may be either reversible or irreversible. 

Applying additivity 

The additive property (1.3) severely restricts the form of the function f{T,Q). 
For instance, if we divide a single-temperature T heat reservoir that absorbs 
heat Q into two identical parts, each still has temperature T and each absorbs 
half the heat <2/2. Since the entropy increment of the composite system is the 
sum of the entropy increments of its parts: f(T,Q) = f(T,Q/2)+ f(T, Q/2) 
or, equivalently, f(T, Q) = 2/(2", <2 / 2) . If instead of dividing the heat reser- 
voir into two identical parts we divide it into n identical parts, then 

f(T,Q) = nf[r^. (1.5) 

There is only one non-trivial solution to (1.5) and that is 

f(T,Q) = g(T)Q, (1.6) 

where g(T) is an as yet undetermined function of the temperature T. (One can 
derive (1.6) by taking the partial derivative of (1.5) with respect to n, and solv- 
ing the resulting partial differential equation.) Accordingly, the entropy of a 
heat reservoir with temperature T changes by 

A S = g{T)Q (1.7) 

when the reservoir absorbs or rejects energy Q through, respectively, cooling 
or heating another object. 

Entropy generation in a two-reservoir, isolated system 

Next we apply these ideas to a composite, two-reservoir, isolated system that 
experiences the following irreversible process: a hotter reservoir directly heats 
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Figure 1 .6 A heat reservoir with temperature T H irreversibly heats a colder res- 
ervoir with temperature T c < '/„ . In this process energy Q is transferred from the 
hotter reservoir to the colder reservoir. 

a colder reservoir and in so doing transfers energy Q from the hotter reser- 
voir to the colder reservoir as illustrated in Figure 1.6. Since the colder reser- 
voir absorbs energy Q and the hotter reservoir rejects energy Q, their entropy 
increments are, respectively, AS C = g(T c )Q and A S H = -g(T H )Q. (Here 
Q > 0.) According to these expressions and the additive property (1.3), the 
entropy increment of this composite two-reservoir system is 

A S = [g(T c )-g(T H )]Q. (1.8) 

If the two heat reservoirs of this composite, isolated system have the same 
temperature T a = T c , then AS = 0. And if their temperatures, T H and T c , are 
different, then heat flows irreversibly from the hotter reservoir to the colder one 
and AS ^ 0. In this way the entropy increment AS quantifies the irreversibility 
of a thermodynamic process of an isolated system. In brief, when the process 
is reversible AS = 0 and when irreversible AS ^ 0. 

Clausius adopted the convention that the entropy of an isolated system 
increases when the system evolves irreversibly in the direction allowed by 
the second law of thermodynamics. Thus, according to Clausius’s convention, 
when T h > T c in (1.8), AS > 0. This convention means that g(T c )~ g(T H ) > 0 
and therefore that g(T ) must be a non-negative, ever-decreasing function of the 
temperature T. 


Reversible heating and cooling 

We can easily overcome the restriction of AS = g(T)Q to heat reservoirs by 
limiting Q to an indefinitely small quantity 5Q. The transfer of an indefin- 
itely small quantity of heat 8Q through heating or cooling must, of course, be 
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reversible, since the direction of that transfer can be reversed by an indefinitely 
small change in the system or its environment. In this case AS = g(T)Q reduces 
to an expression, 

dS = g(T)SQ, (1.9) 

that applies to any thermodynamic system with temperature T that absorbs 
[5Q > 0] or rejects \SQ < 0] an indefinitely small amount of energy 5Q through 
heating or cooling. After all, any finite system acts like a heat reservoir relative 
to an indefinitely small amount of heating or cooling. Again, notice the distinc- 
tion in (1.9) between the differential of a state variable dS and the indefinitely 
small increment SQ. 


Summary 

We pause to summarize our progress thus far. The entropy increment AS is 
additive over a system’s parts. From this requirement it follows that the entropy 
increment is dS [= g{T)8Q\ whenever a thermodynamic system with tempera- 
ture T absorbs [ 5Q > 0] or rejects [ 8Q < 0] energy 5Q through heating or cool- 
ing. The function g(T) is a non-negative, ever-decreasing function of T. 

According to the work of Rudolph Clausius, the entropy of an isolated sys- 
tem remains constant if and only if it evolves reversibly and increases if and 
only if it evolves irreversibly. The content of these statements is encapsulated 
in the entropic version of the second law of thermodynamics: The entropy of 
an isolated system cannot decrease. 

Example 1.2 Joule expansion 

Problem: A partition separates a container into two parts. One part contains a 
gas and the other part is evacuated. The partition is quickly removed and the 
gas fills the whole container in a process called Joule expansion as illustrated 
in Figure 1.7. Does the entropy of the gas increase, decrease, or remain the 
same, and why? 

Solution: According to the entropic version of the second law of thermody- 
namics, the entropy of an isolated system cannot decrease. The container, par- 
tition, and gas constitute an isolated, composite system whose entropy cannot 
decrease. Since the container does not change and the removal of the partition 
is a mere spatial rearrangement of a single object, the entropy of these parts 
of the composite system does not change. Therefore, the entropy of the gas 
cannot decrease; it can only increase or remain the same. Yet the entropy of an 
isolated system remains the same only when it undergoes a reversible change. 


1.6 The Carnot efficiency and Carnot’s theorem 
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Figure 1.7 Joule expansion of a gas into an evacuated chamber. 


And a reversible change occurs only when the direction of the process can be 
reversed by an infinitesimal change in the system or its environment or, alter- 
natively, when the process is quasistatic and without friction or dissipation. 
Neither condition is met here. In particular, the quick removal of the partition 
suggests that this process is not quasistatic. Therefore, the entropy of the gas 
must increase. 


Example 1.3 Humpty Dumpty 

Problem: An egg falls from a second floor window, hits the pavement, and 
breaks. Does the entropy of the egg increase, decrease, or remain the same, 
and why? 

Solution: Again we apply the entropic version of the second law: the entropy 
of an isolated system cannot decrease. Since the egg hits the pavement we 
must include the pavement and the egg in the composite isolated system whose 
entropy cannot decrease. The breaking of the egg is certainly not a reversible 
change since it cannot be reversed by an infinitesimal change in the egg or 
in the pavement. Therefore this process is irreversible and, consequently, the 
entropy of the egg-pavement system increases. It seems reasonable to assume 
that the pavement does not change much. Therefore, most or all of the entropy 
increase occurs in the egg. 


1.6 The Carnot efficiency and Carnot’s theorem 

One consequence of the entropic version of the second law, the entropy of an 
isolated system cannot decrease, is Carnot’s theorem: The most efficient heat 
engine is one that operates reversibly. To see this observe, that Carnot’s sim- 
plest heat engine - one that extracts energy Q H by cooling a hotter reservoir, 
produces work W, and rejects waste energy Q c by heating a colder reservoir as 
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Figure 1.8 The simplest possible heat engine is one that produces work by oper- 
ating between a single hot and a single cold reservoir. When this simplest heat 
engine is reversible it is called a Carnot engine. 


illustrated in Figure 1.8 - can be made part of an isolated system by, say, using 
the work W to raise a weight. The entropy of this isolated system is 

AS=-Q H g{T H ) + Q c g{T c ), (1.10) 

where Q H ,Q C , and W are related, according to the first law, by 

Q h -W + Q c . ( 1 . 11 ) 


The efficiency e of this engine is, by definition, the engine’s benefit, the work 
produced W, divided by its cost, the heat it consumes Q H , that is, 




Qh 


( 1 . 12 ) 


where, given (1.11) and (1.12), we see thatO < e < 1. Using (1.11) and (1.12) to 
eliminate Q H and Q c from the entropy increment (1.10) produces 


g(T a ) 

*(r c )J' 


(1.13) 


Therefore, the larger the efficiency e, the smaller the entropy increment AS. 
Since the entropy of an isolated system cannot decrease, that is, AS > 0, the 
largest possible efficiency e is achieved when AS = 0, that is, when the engine 
operates reversibly. This maximum efficiency of a heat engine operating 
between two heat reservoirs. 


g(T H ) 

g(r c y 


(1.14) 
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is called the Carnot efficiency and the reversible version of the simplest possi- 
ble heat engine is called a Carnot engine. 


1.7 Absolute or thermodynamic temperature 

Neither the entropy increment dS [= g{T)8Q\ nor the Carnot efficiency 
£ camct [=i-s(r H )/*(r c )] are fully quantified because the non-negative, 
ever-decreasing, function g(T) is not yet fully specified. William Thomson 
very cleverly supplied the missing specification in 1848, twenty-four years 
after Carnot proved Carnot’s theorem in 1824 and two years before Rudolph 
Clausius distinguished and harmonized the first and second laws of thermody- 
namics in 1850. 


Temperature 

But first, what is temperature? A thermometer measures temperature, but what 
is a thermometer? A thermometer associates the size of a convenient thermo- 
metric variable, whether the volume, resistance, color, or some other thermo- 
dynamic variable of a system, with a unique number denominated according 
to a temperature scale, whether, for instance, Fahrenheit or Celsius. Consider, 
for example, the mercury-in-glass thermometer illustrated in Figure 1.9. The 
length of the mercury column in the small-bore glass tube is the thermomet- 
ric variable and the way the numbers are arrayed on the side of the glass tube 
defines the temperature scale. 

William Thomson’s idea was to use a Carnot engine, that is, a reversible 
heat engine operating between two heat reservoirs, as a thermometer and its 
efficiency e Camot as its thermometric variable. Of course, a Carnot engine is an 
idealization that can be approached but never realized. After all, a reversible 
heat engine would operate indefinitely slowly and without friction or dissipa- 
tion or, alternatively, in such a way that the slightest change in the system or 
its environment would reverse its operation. Never mind that no one can build 
a perfectly reversible heat engine and that heat reservoirs are idealized objects! 
Simply build the best approximation possible and, if necessary, extrapolate the 
results. 

The idea is simple even if its execution is difficult. First, assign a standard 
temperature T s to some convenient and universally accessible, standard state. 
One such standard state is the triple point of water, that is, the state at which all 
three phases of water are in equilibrium. Second, measure the efficiency £ Camot 
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Mercury »!-« 1 

Figure 1.9 Mercury-in-glass thermometer with (arbitrary) temperature scale. 


of a reversible heat engine operating between two heat reservoirs, one in equi- 
librium with the standard state and the other in equilibrium with the system 
whose temperature T x is to be determined. Third, if T x < T . set 

T\ = 7; (1 - £ fW „ ), (1.15a) 


and if T x > T s set 


Me™,)' 


(1.15b) 


Temperatures T assigned in this way are called absolute or thermodynamic 
temperatures, while temperatures T determined by other kinds of thermom- 
eters are empirical temperatures. 

This algorithm allows one to assign an absolute or thermodynamic tem- 
perature T to any system, in particular to the hot and cold reservoirs between 
which a reversible heat engine operates. In terms of thermodynamic tempera- 
tures assigned according to Eqs. (1.15), the Carnot efficiency of a reversible 
heat engine operating between two heat reservoirs is given by 

£ C amo, =l-= (1-16) 

t h 


where T H and T c < T H are absolute or thermodynamic temperatures. 


Absolute temperature scales 

The essential feature of absolute or thermodynamic temperatures is that the 
ratio of any two temperatures, say T c / T H , T c /t. , or T H /t c , is independent of 
the scale used to denominate the individual temperatures Tc, Th, or Ts. For 
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instance, the SI or Systeme International assigns a standard absolute temper- 
ature Ts = 273.16 K to the triple point of water. In this way there are, to five 
places, 100 kelvin degrees between the normal freezing point of water, 273.15 
K, and the normal boiling point of water, 373.15 K. The SI unit of absolute 
temperature, the kelvin, and its symbol K are, of course, named after Lord 
Kelvin (William Thomson). Engineers in the United States and Canada some- 
times use the Rankine scale, which assigns Ts = 491.688°Ra to the triple point 
of water. In this way the size of a Rankine degree is identical to the size of a 
Fahrenheit degree. 

In the Systeme International all other kinds of temperatures are defined as 
linear functions of kelvin temperatures. Translation from one kind of temper- 
ature to another is straightforward. For instance, to convert 23 degrees Celsius 
to kelvin we simply add 273 to 23 and produce, to three-place accuracy, 296 K. 
And to convert 68 degrees Fahrenheit to Celsius, we subtract 32 from 68 and 
multiply the result by 5/9 to produce 20 degrees Celsius. 


The entropy increment 


The function g(T) that defines the entropy increment dS [= g{'l) <5g] of a sys- 
tem with temperature T that absorbs heat 5Q is constrained by the two expres- 
sions for the Carnot efficiency already encountered: (1.14) in terms of empirical 
temperature and (1.16) in terms of absolute temperature. Thus, 


s{t h ) 

s{TcY 


(1.17) 


that is, Tcg{T c ) = THg{T [1 ). Thus g(l ) — cjT where c is an arbitrary con- 
stant independent of temperature. Choosing c = 1 so that g(T) = l/r and that 
dS= 5Q/T merely fixes the unit of entropy to be that of energy over absolute 
temperature; for instance, in SI units, joules per kelvin. 

While absolute and empirical temperatures are conceptually distinct, main- 
taining two sets of symbols, T and T, serves no practical purpose. For this 
reason we drop the overbar from T and henceforth use the same symbol T for 
both absolute and empirical temperatures. Context indicates the kind of tem- 
perature intended. 


The universality and simplicity of absolute temperatures 

Absolute temperatures are universal in the sense that their definition is inde- 
pendent of the behavior of any one kind or phase of matter. But their most 
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important advantage is that they reduce all thermodynamic expressions, includ- 
ing all equations of state, to simplest form. For instance, the Carnot efficiency 
is no longer £ Camot = 1 - g [T h ) j g (T c ) but rather 

e C ar„o, = 1 (U8) 

And the differential entropy increment dS = g{T)5Q becomes 



In (1.18) and (1.19) and any other expressions derived from these absolute 
temperatures, kelvin or Rankine, must be used. 

That absolute temperatures simplify all thermodynamic expressions includ- 
ing equations of state follows from the crucial role played by the entropy 
increment (1.19) in generating equations of state - a topic we address in the 
next section. For now, reflect on the possibility of expressing the ideal gas 
pressure equation of state in terms of Fahrenheit temperatures. Certainly, 
PV = nR [273 + 5 (T - 32) /9] is less simple, less beautiful, and less useful than 
PV = nRT ! 


1.8 Consequences of the second law 

Our understanding of the entropy increment dS = 5Q/T and of the entro- 
pic version of the second law of thermodynamics, the entropy of an isolated 
system cannot decrease, has important consequences. One of these is, of 
course, Clausius’s second law. After all, the function g{T) was chosen to be a 
non-negative, ever-decreasing function of T in order to make the entropic ver- 
sion of the second law equivalent to Clausius’s version of the second law. 

A second consequence of the entropic second law is Kelvin’s second law: 
An engine whose only result is to cause a single heat reservoir to lose energy 
Q through cooling and perform an equal amount of work W [ = Q > 0] is 
impossible. For the entropy increment AS = —Q/T of this supposed isolated, 
single-reservoir heat engine is negative and therefore violates the entropic 
second law. This argument assumes that the work produced by the engine 
does not change the entropy of the reservoir or its environment. We could, for 
instance, use this work to raise a weight without changing the entropy of any 
system. (Note: In this paragraph and throughout this section the symbols Q and 
W are unsigned, inherently non-negative quantities. Signs indicating whether 
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energy is transferred through heating or cooling and whether work is done on 
or by the system are inserted into the equations.) 

A third consequence is another verbally distinct version of the second law: 
A heat engine can be no simpler than one that extracts energy Q H through cool- 
ing a reservoir with temperature T u , produces work W, and exhausts energy 
Q c through heating a reservoir with temperature T ( . < 7’ ;( . We call this state- 
ment Carnot’s second law because Sadi Carnot emphasized it in his famous 
1824 essay On the Motive Power of Fire. Clearly, a heat engine needs both 
a hot and a cold heat reservoir. For in order that the entropy of the compos- 
ite, isolated system composed of both reservoirs not decrease, the entropy lost 
by the hot reservoir -Q h /T h must be balanced, or more than balanced, by 
the entropy gained by the cold reservoir Q c lT c , that is, -Q H /T H +Q C iT c > 0. 
Thermodynamics texts rarely, if ever, mention Carnot’s second law because 
Carnot did not know that the assertions he made amounted to a new law of 
nature. Furthermore, Carnot’s analysis employed the caloric theory of heat, 
according to which in this context Q H = Q c , rather than the first law of thermo- 
dynamics, according to which in this context Q H =Q C +W . After all, 1 824 was 
at least 20 years before James Joule’s precision experiments of the 1840s began 
to compel acceptance of the first law of thermodynamics. 

While we have used the entropic version of the second law, here and in 
Section 1.6, to derive Carnot’s theorem and the Clausius, Kelvin, and Carnot 
versions of the second law, in actual fact, all four results were articulated before 
Rudolph Clausius formulated the concept of entropy in 1865 - Carnot’s second 
law and theorem in 1824, Clausius’s second law in 1850, and Kelvin’s second 
law in 1851. Any one of these three formulations of the second law, along with 
the first law, is a sufficient foundation for classical thermodynamics. In particu- 
lar, from any one it is possible to prove Carnot’s theorem and also, what earlier 
we merely asserted: A state variable entropy S exists whose increment AS is 
additive over the parts of a composite system. 

Entropy and stability 

Finally, we mention a consequence of the entropic version of the second law 
of thermodynamics of great practical importance. According to the entropic 
second law, the entropy of an isolated system cannot decrease. Therefore, 
if an isolated system is constrained from increasing its entropy, say, for 
instance, by adiabatic walls that separate its hot and cold parts, the system’s 
entropy cannot change, and the system is said to be thermodynamically sta- 
ble. Alternatively, if the entropy of an isolated system can increase because, 
for instance, its hotter parts can heat its cooler parts, the system is said to 
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be thermodynamically unstable. Eventually, when the different parts of this 
system achieve thermal equilibrium, a thermodynamically unstable system 
returns to stability. 

However, the entropy is not always a convenient means of determining 
thermodynamic stability because we often observe or experiment with sys- 
tems that are not isolated. Instead, quite often the temperature and pressure of 
a thermodynamic system are maintained by thermal and mechanical contact 
with an environment. Consider, for instance a liquid in a beaker open to the 
environment. In this case, the crucial question is not whether the entropy of 
such a system can increase but rather whether its Gibbs free energy 

G = E — TS + PV (1.20) 

can decrease. If its Gibbs free energy G can decrease, the system is unsta- 
ble; if its Gibbs free energy G cannot decrease, the system is stable. This 
relation (1.19) of the Gibbs free energy G and the entropy S suggests that a 
minimum in the Gibbs free energy correlates with a maximum in the entropy. 
Both conditions signal stability, but they apply in different circumstances: a 
maximum of the entropy applies to an isolated system and a minimum of the 
Gibbs free energy applies to a system whose temperature and pressure are 
held constant. We will exploit a stability condition based on the Gibbs free 
energy in Chapter 7. 

Example 1.4 Automobile engine efficiency 

Problem: A car engine operates at 37% efficiency but this efficiency is only half 
of its Carnot efficiency. Suppose the effective cold reservoir of a car engine is 
the outside air with temperature 300 K and the burning of the fuel-air mix- 
ture in the piston combustion chamber creates and maintains the hot reservoir. 
What is the absolute or thermodynamic temperature of this hot reservoir? 

Solution: This solution applies the Carnot efficiency £ Cartlot = 1 — T c /T h . 
From the problem statement we know that £ Camot =2x 0.37 and T„ = 300 K. 
Solving for T H we find T H = T c /(l - E Cmml )■ Substituting these values into the 
right-hand side produces T H = 1154 K. 

Example 1.5 The entropy generator 

Problem: The entropy generator consists of a weight of mass m that as it falls 
turns a paddle wheel immersed in a viscous liquid. See Figure 1.10. The liquid 
is in thermal contact with a heat reservoir with temperature T. Assume the 
weight falls quasistatically. How much does the entropy of the viscous liquid 
and of the reservoir change? 
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Figure 1.10 Entropy generator. 

Solution: As the mass m falls quasistatically through a distance h work mgh 
is done on the viscous liquid. This would lead to an increase in the internal 
energy of the liquid and thus to an increase in its temperature were it not that 
the liquid remains in thermal equilibrium with the reservoir. Therefore, as 
the weight falls a distance h the liquid transfers energy mgh in heat to the 
reservoir and the reservoir entropy increases by mgh/T. The entropy of the 
liquid remains constant. If we consider the liquid and the reservoir as a single 
composite system, the entropy generator demonstrates that when work W is 
completely dissipated in a heat reservior of temperature T, the entropy of that 
system increases by W/T. 


1.9 Equations of state 


If we use dS = 5 Q/T to eliminate SQ from dE = 6Q + SW, we produce a 
result, 


dE = T dS+ SW, 


( 1 . 21 ) 


that suggests a relation exists between the internal energy E, the temperature T, 
and the entropy S of a thermodynamic system. Such relations, or equations of 
state, do exist and describe what can be known about a thermodynamic system. 
Equation (1.21) requires that what can be known in this way is consistent with 
the first and second laws of thermodynamics. The entropy state variable S plays 
a key role in this linkage. In particular, equations of state are consistent with 
the first and second laws of thermodynamics if and only if they can be derived 
from the entropy S via Eq. (1.21). 
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The equation dE = TdS + SW is not quite sufficient to demonstrate this 
fact. Recall Example 1.1 in which a piston head compresses a gas enclosed 
in a chamber. The work of compression SW included not only the work on 
the gas but also the work required to overcome the friction between piston 
head and chamber wall. Only when the work SW is performed reversibly, 
that is, quasistatically and without friction or dissipation is SW completely 
described by the thermodynamic variables of the gas system. Similarly, only 
when heat SQ is transferred reversibly between two systems, that is, qua- 
sistatically and therefore reversibly, do these two systems have the same 
temperature. 


Reversible work 

When the work done on a system is reversible, SW = SW rev , and (1.21) 
becomes 

dE = TdS + SW m . (1.22) 

However, there are as many ways of doing reversible work SW m as there are 
kinds of systems. When the system is a simple fluid with scalar pressure P and 
volume V, 

SW aw = ~PdV. (1.23) 

The negative sign ensures that positive, reversible work <5W > 0 on the fluid 
decreases the fluid volume by dV. Simple fluids always push; they never pull. 
When the system is a surface with area A and surface tension a, 

SW lev = a dA. (1.24) 

Positive, reversible work SW m > 0 on the surface increases the surface area by 
dA. Surfaces of this kind always pull; they never push. When the system is a 
paramagnet, the currents that create the externally applied magnetic field B o in 
which the paramagnet is immersed do reversible work 

dW mv — fi o B o dM (1.25) 

by increasing the magnetization M of the paramagnet. 

In each of these examples the reversible work done on a system is a 
product of an intensive variable (pressure P, surface tension a, or externally 
applied magnetic field B o ) times the differential of an extensive variable (vol- 
ume V, area A, or net magnetization M). This factorization of SW iey into an 
intensive variable times an increment of an extensive variable can always be 
accomplished. An extensive variable is one, for instance, that doubles when a 
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system is doubled in size, while an intensive variable remains the same as a 
system is scaled up or down. 


Fluid systems 

Fluid systems are especially important. Gases, liquids, and even solids can be 
treated as fluids in varying degrees of approximation. Combining dE = TdS + 
SW iey and dW m = —PdV produces the fundamental constraint 

dE = TdS - PdV (1.26) 

among the variables of a simple fluid, E, T, S, P, and V, imposed by the first 
and second laws of thermodynamics. We often prefer writing the fundamental 
constraint (1.26) in a form, 

dS = -dE + -dV, (1.27) 

T T 

that emphasizes the role of entropy. For if the entropy is a function of the 
independent variables E and V so that S = S(E,V), then its multivariate differ- 
ential is 


(1.28) 


(Here ( dS/dE) v means the derivative of S with respect to E while holding V 
constant. This use of subscripts in this way, common in thermodynamics texts, 
has the advantage of reminding us, in this case, that E and V are the two inde- 
pendent variables. After all, other pairs of variables can be designated as inde- 
pendent.) Equations (1.27) and (1.28) together imply that 


dS) _ l 

dEj v ~ T 
and 


(1.29) 


g) 


P 

T' 


(1.30) 


Furthermore, the equality of mixed partials, that is, 8 2 S/dVdE = d 2 S/dEdV, 
means that for any fluid 
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where the expressions 1 /T and P/T are themselves functions of E and V 
through (1.29) and (1.30). 


An example 

The equations (1.29) and (1.30) show us how, by taking derivatives of the 
entropy function S (E,V), to derive fluid equations of state that observe the first 
and second laws of thermodynamics. For instance, the entropy function for n 
moles of ideal gas is 

S(E,V)=nRln(VE c '' /nR ) + c (1.32) 


where R is the gas constant, C y is the, presumed constant, heat capacity at con- 
stant volume, and c is a constant independent of all thermodynamic state vari- 
ables. Taking the partial derivatives of S(E,V) we find that (dS/dE) y = C v /E 
and (35/dy)^ = riR/V . Using these in (1.29) and (1.30) produces the two ideal 
gas equations of state: 

PV = nRT (1.33) 


and 


E = 


C V T. 


(1.34) 


In this way, the entropy function (1.32) encapsulates the physics of an ideal 
gas. Consequently, the equations of state, (1.33) and (1.34), satisfy the require- 
ment (1.30) on mixed partial derivatives. 


Working backwards 

But how, in the first place, are entropy functions produced? One way to pro- 
duce an entropy function S(E,V) is to work backwards from known equations of 
state by integrating (8S/dE) T = 1 IT and (SS/dV) E = PIT. Suppose, for instance, 
we start with the equations of state for equilibrium or blackbody radiation 


P 


E_ 

3V 


(1.35) 


and 

E = aVI 4 , (1.36) 


where a is a universal constant called the radiation constant. Blackbody radia- 
tion can be considered a simple fluid because it occupies a volume V and exerts 
a scalar pressure P. Solving (1.35) and (1.36) for PIT and T in terms of E and 
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V allows us to eliminate the former from the right hand sides of ( dS/dE) v = 1 IT 
and (dS/dV) E = PIT. In this way we generate 


(I) 


a 1 / 4 V I/4 

E m 


(1.37) 


and 


dS 'j _ a 1/4 fs 3/4 

dv) E ~ 3V 3/4 


(1.38) 


Integrating (1.37) in the variable E while holding V constant produces 



and, therefore, 

4a 1/ V /4 £' 3/4 

S(E,V) = + /(V), (1.40) 

where in this case the “integration constant” fiV) is an undetermined function 
of V. Similarly integrating (1.38) in V while holding E constant produces 

S(E,V)= 4a ' V 3 E + g(E), (1.41) 

where g(E) is another undetermined function. We determine /(V) and g(E) by 
requiring that the two expressions for the entropy, (1.40) and (1.41), lead to the 
same result. Thus we set the right-hand sides of (1.40) and (1.41) equal to each 
other and find that/(V) = g(E). Therefore, the entropy function of blackbody 
radiation is 


S(E,V) = 


4a 1/ V/ 4 g 3 / 4 

3 


+ b 


where b is a constant independent of both E and V. 


(1.42) 


A summary 

We summarize. The entropy is an additive state variable that is a function of 
the energy E and the other extensive variables of the system. The partial deriva- 
tives of the entropy function in terms of its extensive variables generate the 
equations of state of the system. In particular, (dS/dE) v = 1/7’. Alternatively, 
we can produce the entropy function of a system from its equations of state. 


26 


Thermodynamic entropy 


Entropy could also be expressed in terms of a mixture of extensive variables 
and intensive variables. However, expression exclusively in terms of extensive 
variables E, V, and mole number n, or particle number N, is for a simple fluid 
the most fundamental. For this reason we will emphasize the functions S(E,V) 
and S(E,V,n ) or S(E,V,N). In other texts expect to see entropy expressed in 
terms of a great variety of variables. 

The next section introduces a requirement on the entropy function imposed 
by the third law of thermodynamics. 

Example 1.6 Modifying the ideal gas 

Problem : Suppose we modify the ideal gas pressure equation of state by intro- 
ducing attraction between molecules that reduces the pressure exerted by the 
fluid. We use the parameter a where a is independent of any thermodynamic 
variable to mediate this attraction. Our modification is 

p= nRT _ar^ 

V V 2 ' 

In what way must the energy equation of state, E = C y T, be modified in order 
to make this pair of equations of state consistent with the first and second laws 
of thermodynamics? 

Solution : There is no infallible algorithm for answering this question. We 
start by guessing that the modification takes the form 

E = C v T + f(v), 

where we insist that /(V) is chosen so that the first and second laws of thermo- 
dynamics are satisfied and that C v remains a constant. In particular, we solve 
the two displayed equations of state for 1 IT and PIT in terms of the independent 
variables E and V and find that 1 IT = CJ\E - fiV)\ and PfT = nR/V - aC^i 2 / 
{ Y~\E ~f(V)\ } . Substituting these results into the requirement (1 .3 1) on mixed 
partials produces 

J±L_&J f 3 [”* aC v n 2 j| 
jav|_£-/(v)Jj £ \dE [v V 2 [E-f(V)]\\ v 

Completing these partial derivatives we find that /(V) must satisfy the differ- 
ential equation 
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A solution of this equation is /(V) = ~an 2 /V. Therefore, the modified energy 
equation of state is, up to an arbitrary constant, given by E = CfT - an 2 TV. 


1.10 The third law of thermodynamics 

Energy and entropy are thermodynamically parallel concepts. Each is an addi- 
tive state variable. And the existence of each is implied by the laws of ther- 
modynamics: energy by the first law and entropy by the first and second laws. 
Furthermore, these laws identify ways of measuring their increments. Recall 
that dE = SQ + SW and dS = SQ/T. Thus, one can measure the energy and 
entropy increments A E and AS by carefully controlling and measuring the 
energy transferred to or from a system by heating and cooling or by having 
work done on or by a system. 

Yet energy and entropy play different roles within the science of thermody- 
namics. The energy of an isolated system remains constant, while the entropy 
of an isolated system cannot decrease. The third law of thermodynamics identi- 
fies another behavior of entropy that has no energetic parallel. 

The empirically supported, physical content of the third law is: The entropy 
of a system approaches a definite and finite value that is independent of the other 
thermodynamic variables as its thermodynamic temperature approaches abso- 
lute zero. Max Planck (1858-1948) added to this empirical content an extremely 
useful convention: The entropy S — > 0 as the thermodynamic temperature T — > 0. 
A careful analysis of Planck’s convention reveals that it actually consists of two 
statements: (1) the entropy of all systems approach the same constant as T — » 0, 
and (2) the entropy approached is S = 0. The empirical content of the third law 
plus Planck’s convention sum to the following version of the third law: The 
entropy of every thermodynamic system approaches zero as its thermodynamic 
temperature approaches absolute zero. Symbolically, S —*■ 0 as T — > 0. 

The physical content of the third law constrains the low temperature behav- 
ior of thermodynamic systems. Furthermore, Planck’s convention makes it 
possible to define the absolute entropy of a system and so to produce tables 
of absolute entropies. The third law also has an interesting quantum statistical 
interpretation - a subject we take up in Chapter 4. 


An illustration 

Here we simply observe whether or not the entropy of various model systems 
obeys the third law of thermodynamics, that is, whether or not S — * 0 as T — > 0. 
For example, the entropy of an ideal gas is, in its most general form. 
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S{E,V)=nR\nV + C v \nE + c, (1.43) 

where c stands for a constant independent of the any thermodynamic variable. 
We use the energy equation of state E = CJT to eliminate E from the entropy 
function (1.43) in favor of T. Doing so produces 

S(T,V) = nR\nV + C v \nT + c, (1.44) 

where the constant c has been redefined. As T — > 0, this function diverges to 
negative infinity. Thus, there can be no, conventionally chosen, finite value of 
the constant c that makes S — * 0 as T — > 0. Therefore, the entropy of an ideal 
gas violates the third law of thermodynamics. 

However, the ideal gas and other non-third-law-compliant models remain 
useful in high temperature regimes. If a model system violates the first or sec- 
ond laws of thermodynamics, it is useless and we discard it. But if a model vio- 
lates the third law of thermodynamics, we simply recognize its limited validity. 
For this reason the third law of thermodynamics is neither as foundational nor 
as consequential as the first and second laws. Even so, the empirical content of 
the third law is invariably observed and cannot be derived from the other laws 
of thermodynamics. 

Example 1.7 The third law and blackbody radiation 

Problem: Determine whether or not the blackbody radiation equations of state 
obey the third law of thermodynamics. 

Solution: The blackbody radiation equations of state are P = EI3V 
and E = aP. According to Eq. (1.42) these imply the entropy function 
5(£,y) = h + 4a I/4 y 1/4 £; 3/4 /3 where b is a constant independent of E and V. 
Using the energy equation of state E = aT* to eliminate E in S(E,V) in favor 
of the temperature T produces S(T,V) = b + AaV'^T /3 . Therefore, S — > b as 
T — ► 0 and the constant b can be chosen to be zero. The blackbody radiation 
equations of state obey the third law. 


Problems 

1.1 Compressing a fluid 

A weight falls on a piston and irreversibly and adiabatically compresses the 
fluid system contained in the piston chamber (Figure 1.11). Does the entropy 
of the fluid system increase, decrease, or remain the same and why? 


Problems 


29 


T 



Figure 1.11 A weight falls on piston that irreversibly compresses a gas. See 
Problem 1.1. 

1.2 Derivation 

Derive Eqs. (1.6) from (1.5). (See hint in text.) 

1.3 Heat capacity 

An object with constant heat capacity C [= 8Q/dT] absorbs heat and its tem- 
perature rises from 7’ to T f . What is its entropy increase? 

1.4 Isothermal compression of ideal gas 

Consider an ideal gas whose pressure equation of state is PV = nRT and whose 
internal energy E = E[T ) is an unknown function of temperature. What is the 
increase in its entropy AS [= S f - S t ] as the gas volume is changed from an ini- 
tial value V t to a final value V f while keeping the temperature T constant? (Hint: 
Integrate dS = dE/T + P dV/T.) 


1.5 Entropy increment 

An ideal gas whose equations of state are PV = nRT and E = 5nRT/2 evolves 
quasistatically from state A to state B to state C as shown in Figure 1.12. Find 
the net entropy increment AS = S c - S A per mole of ideal gas. (Hint: Use the 
graph to integrate dS = dE/T + PdV/T.) 

1.6 Valid and invalid equations of state 

In the following fluid equations of state the symbols a and b stand for positive 
constants that are independent of all thermodynamic variables. 
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Figure 1.12 The quasistatic path taken by an ideal gas. Used in Problem 1.5. 

(1) PV = aT and E = bjf 

(2) P = E/V and E = bT 

(3) P = aT and E = bTV 

(4) P = aT sin(£ / V) and E = -aT cos(bEV) 

(5) P = aE and E = bT 

(a) Identify which two of these five pairs of equations of state do not obey the 
first and second laws by identifying which do not observe the equality of mixed 


(b) Derive the entropy function of the three pairs of equations of state that do 
obey the first and second laws. 

1.7 Entropy function 

An entropy function of a simple fluid takes the form S [E, V) = aEV 2 where 
a is a positive constant. Derive the two equations of state of this hypothetical 
simple fluid. 


1.8 Room-temperature solid 


One version of the entropy function of a room-temperature solid is 



where a Po , k T o , C v , E a , and V u are positive constants whose values characterize 
the system, (a) Find the energy E and pressure P of the room-temperature solid 
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as functions of V and T. (b) Does this system observe the third law of thermo- 
dynamics? Why or why not? 


1.9 Valid and invalid entropy functions 

Some of the following entropy functions S(E,V) for a simple fluid do not result 
in a positive thermodynamic temperature, some do not observe the third law 
of thermodynamics, some violate both requirements, and some violate neither 
requirement. Identify which of the following entropy functions are not valid 
and indicate the reason or reasons for their invalidity. The constants a > 0 and 
b > 0 and are independent of all thermodynamic variables. Also P > 0 and 
V>0. 


(1) 

S(E,V) = 

b \n(EV) 

(2) 

S(E,V) = 

b InCEV 2 ) 

(3) 

S(E,V) = 

b InCV/E 2 ) 

(4) 

S(E,V) = 

b In (E/VO 

(5) 

S(E,V) = 

a(EV) 

(6) 

S(E,V) = 

a exp (-EV) 

(7) 

S(E,V) = 

--avJE 
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2.1 Boltzmann and atoms 

The thermodynamic view of a physical system is the “black box” view. We 
monitor the input and output of a black box and measure its superficial char- 
acteristics with the human-sized instruments available to us: pressure gauges, 
thermometers, and meter sticks. The laws of thermodynamics govern the rela- 
tions among these measurements. For instance, the zeroth law of thermodynam- 
ics requires that two black boxes each in thermal equilibrium with a third are in 
thermal equilibrium with each other, the first law that the energy of an isolated 
black box can never change, and the second law that the entropy of an isolated 
black box can never decrease. According to these laws and these measurements 
each black box has an entropy function S(E,V, ...) whose dependence on a small 
set of variables encapsulates all that can be known of the black box system. 

But we are not satisfied with black boxes - especially when they work well! 
We want to look inside a black box and see what makes it work. Yet when we first 
look into the black box of a thermodynamic system we see even more thermody- 
namic systems. A building, for instance, is a thermodynamic system. But so also 
is each room in the building, each cabinet in each room, and each drawer in each 
cabinet. But actual thermodynamic systems cannot be subdivided indefinitely. 
At some point the concepts and methods of thermodynamics cease to apply. 
Eventually the subdivisions of a thermodynamic system cease to be smaller ther- 
modynamic systems and instead become groups of atoms and molecules. 

The Austrian physicist Ludwig Boltzmann (1844-1906) thought in terms 
of atoms and molecules at a time when their reality could still be questioned. 
The distinction between atoms and molecules was not always important to 
Boltzmann, but he insisted on their reality. In particular, Boltzmann related the 
entropy of a thermodynamic system to the possible dispositions of its atoms 
and re-interpreted the second law of thermodynamics in terms of the number of 
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its possible atomic arrangements. In doing so Boltzmann helped create the sta- 
tistical approach to thermodynamics. But these achievements cost Boltzmann 
much effort and were in his day strongly resisted. 

While the concept of an atom as an indivisible, ultimate particle originates 
in the fifth century BCE with the Greek thinkers Leucippus and Democritus, 
the first empirical evidence of their existence came much later, around 1800, as 
a result of Joseph Proust’s (1754-1826) and John Dalton’s (1766-1844) inter- 
pretation of the way chemical elements combine into compounds. But neither 
Proust’s and Dalton’s interpretation nor Boltzmann’s speculations compelled 
a belief in atoms. 

Ernst Mach (1838-1916), an influential German-speaking contemporary 
and some-time colleague of Boltzmann, flatly rejected the existence of atoms. 
According to Mach and those who adopted his philosophy of science, atoms 
add little or nothing to our understanding of the world. The goal of science is 
to correlate directly observable quantities as precisely and as economically as 
possible. In this sense thermodynamics is the perfect science because thermo- 
dynamics achieves this goal without invoking the existence of invisible entities. 
Given what was known and not known at the time, Boltzmann found Mach’s 
criticism difficult to counter. 

Boltzmann’s position was not helped by his inadequacies as a writer. Although 
he was an outstanding scientist, Boltzmann’s books and papers are neither con- 
cise nor clear. Rather, they probe intuitively and in the process mask important 
assumptions. Thus Boltzmann invited criticism. And that his fife’s work was 
challenged rather than affirmed troubled him. At the age of 62 on September 5, 
1906, beset by oncoming blindness and periods of depression, Boltzmann took 
his own fife. Boltzmann was unaware at the time that his long struggle to intro- 
duce atoms into physics had been vindicated. For Albert Einstein (1879-1955) 
in 1905 had shown in a paper on Brownian motion that the random motions of 
small but visible particles suspended in water were a direct consequence of the 
random motions of yet smaller invisible atoms and molecules. 

While Boltzmann’s path of discovery took him through his celebrated 
H-theorem, our path exploits the insights of those who since Boltzmann’s death 
have reflected on the meaning of his work. Even so, in this chapter we remain 
faithful to ideas that build directly upon Boltzmann’s contributions. 


2.2 Microstates and macrostates 

When Boltzmann visualized the interior of a thermodynamic system he saw 
not only atoms and molecules but also their possible arrangements. But what is 
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Figure 2.1 The possible microstates of a three-particle, two-sided system. 

an arrangement or as we now say a microstatel (The German word Boltzmann 
used is sometimes translated as complexion.) 

Consider a simple system that contains only three particles. We represent 
these particles, in Figure 2.1, with the symbols: •, o, and <g>. They could be 
atoms of three different elements or could be identical particles that are in 
some other way distinguishable. The system is divided into two equal- volume 
parts, left and right sides, between which the particles may move freely. While 
simple, this system is just complex enough to illustrate the distinction between 
a system, its macrostates, and their microstates. 

A microstate is identified by a detailed particle-level description of the sys- 
tem. Because each of the three particles of this system may be in one of its two 
sides, this system may realize 2 3 = 8 different microstates. Each one of these 8 
microstates is illustrated in Figure 2.1. 

A macrostate consists of a set of microstates that can be described with a 
relatively small set of variables. In more realistic contexts a macrostate is a 
thermodynamic or equilibrium state and the variables in terms of which it is 
described are the thermodynamic variables E,V,N, .... Consider the macro- 
state of the three-particle, two-sided system in which any two particles are in 
the right side of the volume and the remaining particle is in the left. We see 
from Figure 2.1 that this macrostate consists of just three microstates - those 
illustrated in the second row of the figure. This number of microstates can 
also be calculated by choosing two particles (for the purpose of putting them 
in the right half) from among the three available. We may choose the first 
particle in three ways and the second in two ways from among the remain- 
ing particles for a total of6or3! = 3x2xl ways. Since the order in which 
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the two particles are placed into the right side is irrelevant for the purpose 
of counting microstates, we divide 6 by the number of ways of ordering two 
particles, that is, by 2 or 2!=2 x 1. In this way we find the number of micro- 
states in the macrostate in which two of three particles occupy the rightmost 
chamber to be 3 or 3 !/(2! 1 !). 


Generalizations 

This simple example can be generalized in several ways. If the system contains 
N particles instead of 3, and still has two different sides that can be occupied by 
each particle, the binomial coefficient AM/[n ! ( A - n) !] , that is, “N choose n” 
identifies the number of ways in which it is possible to distribute N distinct 
items between the two sides, one with n particles and the other with N - n 
particles. 

Alternatively, if a particle on the left has energy e and a particle on the right 
has energy 2s and we constrain the total energy E of the three-particle system 
to be 4e, then the three microstates illustrated in the third row of Figure 2.1 
compose this macrostate. 


Distinguishable classical particles 

In describing these microstates and macrostates we have distinguished each 
particle from all the others - graphically so in Figure 2.1. Classical particles 
can always be distinguished from one another. This is not because classical par- 
ticles necessarily have individually distinguishing features, but rather because 
their different trajectories through space and time remain distinct. 

Imagine, for instance, that you display two apparently identical steel ball 
bearings to a friend and have your friend cover, and then, after a few seconds, 
uncover their eyes. While your friend does not know if you have exchanged 
the two ball bearings, you do know because you have kept your eyes open and 
never lost track of their different trajectories. Ball bearings are models of clas- 
sical particles and identically machined ball bearings are models of identical 
classical particles. 

It would have been natural for Maxwell, Boltzmann, and other nineteenth- 
century physicists to suppose that the invisible, sub-microscopic world of atoms 
and molecules is simply a smaller version of the visible, macroscopic world. 
Indeed, Boltzmann argued that the ultimate particles, atoms and molecules, are 
always distinguishable one from another. In this chapter and the next we adopt 
this classical assumption that identical particles, for instance, populating a 
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container of helium gas, are just as distinguishable from each other as ball bear- 
ings and for the same reason, namely, that we can, at least according to the prin- 
ciples of classical physics, follow their trajectories with indefinite precision. 


2.3 Fundamental postulate 

According to the fundamental postulate of statistical mechanics: the micro- 
states of an isolated thermodynamic system are equally probable. Since the 
simple three-particle, two-sided system illustrated in Figure 2. 1 has 8 micro- 
states consistent with its description and each, according to the fundamental 
postulate, is equally probable, the probability that this system realizes any one 
of these 8 microstates is 1/8. Furthermore, because the macrostate with exactly 
two particles on its right side is realized in 3 out of 8 equally probable micro- 
states, the probability of this macrostate is 3/8. 

Multiplicity 

The assumed equal probability of the microstates of an isolated system is the 
material out of which we compose the probability of a macrostate. According 
to the fundamental postulate, the probability of a macrostate is proportional 
to the number of microstates contained in a macrostate. Boltzmann called the 
number of microstates that compose a macrostate its Permutibilitdt. We use the 
English word multiplicity and denote multiplicity with the symbol £1 


Limitations and counter-examples 

The fundamental postulate does not apply to all isolated systems but only to 
those whose microstates are defined in such a way that their equal probability 
is plausible. Consider, for instance, a single particle contained within a two- 
dimensional, square box as shown in Figure 2.2. The particle reflects smoothly 
and elastically from the sides of the box in such a way that it periodically 
retraces its path. When we superimpose a uniform grid that identifies the spa- 
tial microstates available to this two-dimensional, single-particle system, we 
see that its microstates are not all equally probable. In fact, most of the system 
microstates are never occupied. Furthermore, adding more particles to this box 
may not help realize the fundamental postulate unless the particles are allowed 
to interact and even then special initial conditions may exist for which not all 
the possible microstates are realized with equal probability. 
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Figure 2.2 A particle reflects smoothly and elastically from the walls of a two- 
dimensional box and periodically retraces its trajectory. The superimposed grid 
identifies the microstates available to this one -particle system. 


Counter-examples to the fundamental postulate involve special initial con- 
ditions of a system with only a few particles. In contrast, the initial conditions 
of a typical thermodynamic system are unknown and these systems, such as 
a cubic centimeter of air, contain several times 10 19 particles. Their structure 
and evolution is so complex one cannot keep track of the many microstates 
they occupy. We adopt the fundamental postulate as a way of quantifying our 
partial knowledge of the state of such a system. In particular, we adopt the 
fundamental postulate in order to avoid biasing the assignment of probabilities 
when we have no reason to prefer one microstate over another. Still, the fun- 
damental postulate has not been derived from yet more fundamental principles 
and cannot be directly tested. Instead, one adopts the postulate of equiprobable 
microstates as a plausible hypothesis from which one derives consequences 
that can be tested. 


Thermodynamic systems 

We call a system to which the fundamental postulate applies a thermodynamic 
system. In general, thermodynamic systems are equilibrium or steady-state 
systems in the sense that they can be described in term of thermodynamic 
variables that do not change with time as long as the environment of the sys- 
tem does not change in time. Thermodynamic systems can always be described 
with a few thermodynamic variables and their macrostates can always be char- 
acterized by their multiplicity. 
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Improbable macrostates 


The fundamental postulate implies that an isolated thermodynamic system 
moves probabilistically and without bias among all its possible microstates. 
But if so, a system may also move among sets of microstates that correspond 
to unusual macrostates, for instance, to air rushing to one comer of a room or 
to a cup of hot tea becoming yet hotter on a cold afternoon. We never observe 
such counter-intuitive macroscopic evolutions, not because they are impossible 
but rather because they are extremely improbable. The multiplicities of such 
unobserved counter-intuitive macrostates are overwhelmingly small compared 
to the multiplicities of the macrostates that are invariably observed. 


Example 2.1 Uniform density 


Problem: Each particle in an /V-particle gas may move freely between right and 
left chambers of equal volume. Because these chambers have equal volumes, 
the probability that any one particle resides in any one chamber is 1 out of 2 
or 1/2. In this way, each distinct division of the N distinguishable particles 
between the two chambers models a single microstate each one of which is 
equiprobable. Show that the most probable macrostate is one in which the two 
chambers contain equal numbers of particles. 

Solution: The probability P(ri) of a macrostate with n classical, and so dis- 
tinguishable, particles in the rightmost chamber and N~n particles in the left- 
most chamber is 



where the binomial coefficient N ! / /[n!(/V-n)!] is the multiplicity O of this 
macrostate and (l/2) is the probability of any one microstate of this macro- 
state. A few empirical tests for small values of N suggest that the value of n that 
maximizes Pin) is approximately N/2 - in which case the two chambers have 
equal numbers of particles. 

A more systematic derivation of this result is possible when the numbers of 
particles in the two chambers, n and N~n, are so large as to he effectively contin- 
uous variables, that is, when n » 1 and N - n » 1 . In this case Stirling’s approx- 
imation n ! ® n n e~" 'Jinn or, more crudely, In n ! = n In n - n , obtains and allows 
us to use ordinary calculus to maximize, not P(n), but rather In Pin). Then 
In P(n) = NlnN — N -n\nn + n-(N — n) ln(/V — n) + (iV — n) — /V In 2 
= N\nN -nlnn-(N -n)\n(N -n)-Nln2. 
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Setting the derivative of In P (n) with respect to n equal to zero produces 
- In n + ln(A - n) = 0, 


whose solution is n = N/2. 


2.4 Statistical entropy and multiplicity 

It may be surprising that a statistical version of the entropy is not given to us 
ready-made but, rather, must be constructed out of desirable properties and the 
laws of thermodynamics. To begin this construction, consider the irreversible 
processes illustrated in Figure 2.3. In the first sequence the particles of a gas 
are confined to one half of a space by an impenetrable barrier, the barrier is 
removed, and the gas particles fill the space available. In the second, two iden- 
tical bodies with different temperatures, T H and T c < T w are separated by an 
adiabatic barrier, the barrier is removed, and the two bodies approach thermal 
equilibrium. 


Dependence of entropy on macrostate multiplicity 

In each of these irreversible transitions the entropy of a macrostate of an iso- 
lated system increases from an initial value S. to a final value S f > S. in accor- 
dance with the second law of thermodynamics. At the same time the number of 
microstates accessible to the isolated system increases from an initial value £2, 
to a final value il f > £2 ; . This correlation between the evolution of the entropy 
S of an isolated system and the evolution of its multiplicity £2 suggests that the 
statistical entropy of a macrostate is an ever increasing, monotonic function 
S (£2) of macrostate multiplicity. Assuming that such a function exists is the 
first step in constructing a definition of statistical entropy. 

Boltzmann, in his landmark 1877 paper, suggested that the function S (£2) 
is adequately expressed by the proportionality S In £2. Later, Max Planck 
(1858-1947), starting from reasonable hypotheses, derived the relation, 
S = k In £2, where k is a universal constant. Here we follow and at the same time 
generalize Planck’s construction. While our derivation is lengthy, we emerge 
with a result only slightly more complex than Planck’s - that the entropy of an 
isolated system in a given macrostate is S (£2) = c + kin £2 where £2 is the mac- 
rostate multiplicity, k is a universal constant, and c is a constant that ensures 
that the entropy is an additive state variable. 
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Figure 2.3 Top sequence: an impenetrable barrier is removed and the gas fills its 
container. Bottom sequence: an adiabatic barrier is removed and the two parts 
approach a common temperature. 


Additivity 

Additivity is the second property we incorporate into our definition of statisti- 
cal entropy. Accordingly, the entropy S' of a two-part composite system, illus- 
trated in Figure 2.4, in which each part is itself an isolated system, denoted A 
and B, is additive in the sense that the entropy of the composite system is the 
sum of the entropy of its parts. Thus, 

S A+B =S A +S B , (2.1) 

where S A + B is the entropy of the composite system, S A is the entropy of sub- 
system A, and S B is the entropy of sub-system B. Adopting additivity (2.1) 
ensures that the statistical entropy so constructed will be consistent with the 
requirement, imposed in Section 1.5, on the additivity of the increments of 
thermodynamic entropy. The notation of (2.1) allows for sub-systems that, in 
general, may be quite different from one another as in, for example, a system 
composed of a bottle of helium gas in one corner of a laboratory and a crystal 
of salt in another. 

Given that the statistical entropy is a function S (Q) of macrostate multiplic- 
ity O, additivity (2.1) implies that 

S A+fi (Q A+s ) = S A (Q A ) + S a (Q B ), (2.2) 

where Q A is the multiplicity of a macrostate of sub-system A, Q B is the mul- 
tiplicity of a macrostate of sub-system B, and Q A+B is the multiplicity of a 
macrostate of the composite system A+B. When the two subsystems are sim- 
ply smaller copies of the composite system, the functions S A ( Q A ) and S B (fl B ) 
and the multiplicities il A and Q B must be similarly structured and additivity 
reduces to extensivity. 
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A + B 

Figure 2.4 Isolated composite system A+B consisting of two isolated 
subsystems, A and B. 


Independence 

Thirdly, we assume that isolated sub-systems, A and B , independently realize their 
macrostates. Therefore, the multiplicity of the composite system is given by 

□ A+B = Q. a Q b (2.3) 

and condition (2.2) becomes 

S a+b (Q a Q b ) = S a (Q a ) + S b (Q b ). (2.4) 

Actually, the very isolation of the two subsystems implies the independence of 
their macrostates. This result (2.4) incorporates all three of the following con- 
ditions: the dependence of entropy on macrostate multiplicity S (Q), additivity 
(2.1), and the independence of isolated subsystems (2.3). 

Exploiting derivatives 

We now take derivatives of expression (2.4) with respect to the macrostate 
multiplicities Q A and Q B - a tactic that reduces a stronger condition (additivity 
of entropies) to a weaker one (additivity of entropy increments). If we want to 
restore the full content of additivity, we must reimpose (2.4) on the result of 
our derivation. The two partial derivatives of (2.4) yield 

S' A+B (£2 a Q b )£2 b = S' A (£2 a ) (2.5a) 

and 

S A+B (Q a Q b )Q a = S' B (Q b ), (2.5b) 

where a prime symbol indicates a derivative with respect to the indicated argu- 
ment. For example, f'(x) = df /dx. Eliminating S' A+B (Q a Q b ) from (2.5a) and 
(2.5b) produces 


n A s' A (n A ) = o. B s' B ( Q b ). 


(2.6) 
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Since the variables £2 A and Q B realize their values independently, Eq. (2.6) 
implies that 

Q A S^(Q A ) = k (2.7a) 

and 

Q B 5;(£2 B ) = Jt, (2.7b) 

where, formally, k is a separation constant. Integrating (2.7a) and (2.7b) 
produces 

S A (Q A ) = c A +klnQ A (2.8a) 

and 

S B (Q B ) = c B +fclniI B , (2.8b) 

where c A and c B are integration constants. 

The meaning of “constant” 

We pause to consider the nature of the three constants, k, c A , and c B , generated 
by solving the coupled differential equations (2.5a) and (2.5b). The word con- 
stant should always prompt us to ask: “Constant with respect to what?” In this 
case, the answer is: “Constant with respect to the variables in terms of which 
the derivatives in Eqs. (2.5)-(2.7) are defined, that is, constant with respect to 
the macrostate multiplicities Q A and Q B .” 

The constant k is additionally constrained. Since k is associated with both 
sub-systems A and B (actually, any number of subsystems), k is independent of 
the parameters that identify any particular macrostate of any particular system 
or sub-system. In brief, k is a universal constant. Max Planck was the first to 
emphasize the universality of k. In his honor H. A. Lorentz (1853-1928), for a 
time, called k Planck’s constant. However, the phrase Planck’s constant is now 
reserved for another of Planck’s discoveries, and we refer to k as Boltzmann ’s 
constant. We shall find that k = R/N A , where R is the misleadingly named “gas 
constant” and N A is Avogadro’s number. In SI units the value of Boltzmann’s 
constant is given by 

k = 1.38xl0~ 23 kg m (2.9) 

Ks 

Thus, the units of Boltzmann’s constant k are those of energy over thermody- 
namic temperature, that is, k has the units of entropy. 


2.4 Statistical entropy and multiplicity 
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The statistical entropy of an isolated system 

If Eqs (2.8a) and (2.8b) are true of any isolated system, then 

5 A + B( Q A +B ) = C A +B + ^ ln£2 A + B (2.1.0) 

applies to a composite system composed of sub-systems A and B. Additivity 
(2.2) and independence (2.3), or equivalently (2.4), then require that the three 
constants, c A+B , c A , and c B , are related by 

c A+B =c A +c B . (2.11) 

Therefore, when we apply the general result 

S(Q) = c + A:lnQ (2.12) 

to a composite system and to each of its two isolated parts, we should always 
be ready to adopt (2.1 1) in order to restore the full content of additivity to our 
definition of statistical entropy. 


Alternate forms 

We can express the general result (2.12) in a revealing form by writing it twice, 
once for each of two different macrostates of the same system, an initial state 
denoted i and a final state denoted/, and subtracting the two equations from 
each other. In this way we find that 

S (Q f ) = c + & In Q, (2.13) 

and 

5(Q / } = c + itlEQ / (2.14) 

imply 

5(Q / )-S(Q ( ) = ikln^j. (2.15) 

This result (2.15) is especially useful in describing the entropy increment of an 
isolated system undergoing an irreversible change. 

In particular, Eq. (2.15) allows us to restate the second law of thermody- 
namics in the language of multiplicity: Because an isolated system may relax 
but may never impose an internal constraint, the multiplicity of the final mac- 
rostate of an isolated system is at least as large as the multiplicity of its ini- 
tial macrostate. Some texts characterize the irreversible transition from an 
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internally constrained system to one less constrained as one from a relatively 
ordered macrostate to one that is less ordered. Actually, Boltzmann never used 
the words order and disorder (German: Ordnung and Unordnung ) in his 1877 
paper. And, when applied to the relative entropies of a system, the opposites 
order and disorder sometimes mislead. 

Alternatively, we can replace the initial macrostate of (2.15) Q, with a spe- 
cial reference macrostate Ll n and the final macrostate multiplicity Q / with an 
arbitrary macrostate multiplicity 12. Then (2.15) assumes a form. 



(2.16) 


useful for describing the entropy S (12) of an arbitrary macrostate relative to the 
entropy S (!2 o ) of a special reference macrostate. 


The entropy of a non-isolated system 


By carefully deriving (2.12), (2.15), and (2.16) we discover exactly what ideas 
are contained in these expressions for statistical entropy and what ideas are 
not. In particular: the statistical entropy of an isolated system is a function of 
macrostate multiplicity, is an additive function of state, and reflects the inde- 
pendence of isolated parts. 

Recall, though, a basic thermodynamic truth: everything known about the 
thermodynamic behavior of a system including all its equations of state is 
encapsulated in how its entropy depends upon its extensive variables. This is 
so even when the special assumptions used (isolating the system and positing 
the existence of independent subsystems) in order to determine how the statis- 
tical entropy depends upon macrostate multiplicity are not observed. 

For instance, in the applications we explore throughout much of this book 
we consider a closed, A-particle, simple fluid system with extensive variables: 
internal energy E, volume V, and particle number N. However, since the num- 
ber of particles A in a closed system is conserved, its macrostates are a func- 
tion only of E and V. In this case, 12 = 12 (fc, V, N) ,c = c(n), and the general 
result (2.12) can be expressed as 


S(E, V, N) = c[N) + k\nQ(E, V, N). 


(2.17) 


Thus, once we determine the multiplicity function 12 (E, V, N) and, by requir- 
ing additivity, the “constant” c(iV), we know the entropy function S (E, V, N) . 
And once we know the entropy function we know, through its derivatives, the 
system’s equations of state. Furthermore, these equations of state apply in all 
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situations - not only when the system is closed and isolated. They describe, for 
instance, how the system changes when heated or cooled, when performing 
work or being worked upon. To summarize: The presumption of an isolated 
system allows us to determine the entropy function and once we know this 
function we need no longer maintain this presumption. 

The limits of classical statistical thermodynamics 

The arguments of this section do not allow us to determine the absolute entropy 
of any macrostate. An expression for absolute entropy is one that, given com- 
plete information about a particular macrostate, provides a determinate value 
of the entropy of that macrostate. Equations (2.15) and (2.16) obviously pro- 
duce only relative entropies. And only relative entropies emerge from the first 
and second laws of thermodynamics. 

This limitation to relative entropies originates with our inability, given clas- 
sical presuppositions, to uniquely identify and count microstates. Furthermore, 
reference microstates in classical statistical mechanics are necessarily arbitrary 
and so classical macrostates are also necessarily arbitrary. Only the introduction 
of quantum physics allows us to uniquely identify microstates. But, of course, 
Boltzmann and the other classical statistical mechanicians were unaware of the 
possibilities inherent in quantum physics. 

What Boltzmann did know was how to arbitrarily discretize space in such 
a way as to construct the arbitrary microstates we call classical microstates. 
Boltzmann then applied the fundamental postulate to these arbitrarily con- 
structed classical microstates by asserting that S °= InQ. Boltzmann’s method 
of discretizing phase space allowed him and will allow us to determine clas- 
sical versions of statistical entropies from which it is possible to derive equa- 
tions of state. Of course we will use S (£2) = c + k In Q , in place of Boltzmann’s 
S °= In Q in order to ensure that the entropy is always an additive function of 
state variables. 


Example 2.2 Joule expansion 

Problem: Consider, as in Example 2.1, a system with left and right, equal- 
volume chambers containing a total of N identical but classical, and so dis- 
tinguishable, particles. Suppose this system is initialized in a macrostate in 
which all its particles are contained within its left chamber. Then the par- 
ticles are allowed to move freely between both chambers and achieve a final 
equilibrium macrostate in which the particles are distributed throughout both 
chambers as illustrated in Figure 2.5. How much does the entropy of the sys- 
tem increase? 
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Figure 2.5 Joule expansion. After the barrier that keeps the particles in the left 
hand side of two equal-volume chambers is removed, the particles occupy both 
chambers. 


Solution'. The number of spatial microstates available to a single parti- 
cle increases by a factor of 2 during this process. And the number of spa- 
tial microstates available to two distinguishable particles increases by 2x2. 
Therefore, the multiplicity of the A-particle system increases by 2 N during this 
Joule expansion. Consequently, the ratio of the final to the initial macrostate 
multiplicity is 


Qy 

Q, 


2 N , 


and the entropy is incremented by 


S.-S, =k\n — 2-1 


■&) 


= Nk In 2. 

Apparently each particle contributes k In 2 to the entropy increment. 


Example 2.3 A paradox 

Problem: Consider a system of 2 N identical, classical particles initialized with 
N particles in the left of two equal-volume chambers and N in the right. The 
particles are allowed to move freely and achieve a final, equilibrium state. 
What is the entropy increment? 

Solution: One response is to say that the entropy of the N particles initially 
in the left chamber increases, as in Example 2.2, by Nk In 2 as those particles 
spread throughout both chambers and likewise for the particles initially in 
the right chamber. In this way the total entropy would increase by 2Nk\n2. 
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But this is absurd. For, if we were then to reinsert a barrier between the two 
chambers, remove that barrier, and allow the particles on each side to again 
mix with each other, the system entropy would again increase by 2Nk In 2 
and so on. 

The key to resolving this paradox is to recall that a system’s entropy only 
increases when the multiplicity of its final macrostate is larger than the multi- 
plicity of its initial macrostate. In this problem the initial and final macrostates 
are the same because they have the same description: N classically identical 
molecules occupy each chamber. Therefore the initial and final macrostate 
multiplicities and entropies are the same, and there is no entropy increment. 
True, the final microstate of the system may differ from its initial microstate, 
but this is of no consequence for entropy. The entropy increment is a function 
only of macrostate multiplicity and not of the particular microstates that com- 
pose the macrostate. 


Example 2.4 Entropy of mixing 

Problem: A system consisting of 2 N classical particles is initialized in a mac- 
rostate in which N particles of one kind, for example, nitrogen molecules, are 
all in the left of two equal- volume chambers and N particles of another kind, 
for example, oxygen molecules, are all in the right chamber. The particles are 
allowed to mix. What is the entropy increment? 

Solution: The microstates available to each kind of particle are doubled dur- 
ing this process and this doubling does increase the number of microstates 
available to the system in its final macrostate. Therefore, the initial and final 
multiplicities of the 2/V-particle system are in the ratio 



Accordingly the entropy increment AS = k ln(Q^/£2 ( ) of the system is 
AS = 2Nk In 2. 


2.5 Maxwell’s demon 

While Boltzmann was the first to suggest that a relation exists between the 
entropy S of an isolated system and its macrostate multiplicity Q, it was James 
Clerk Maxwell (1831-1879) who first suggested that entropy is a probabilistic 
concept - first privately in a letter to his friend Peter Guthrie Tait in 1867 and 
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Figure 2.6 Maxwell’s demon. By opportunely opening and closing a door con- 
necting two chambers of gas, the demon can increase the temperature of the gas in 
one chamber over that of the other. 

later in his 1871 text Theory of Heat. Maxwell died before he could develop 
this idea, but his way of arguing for it had important consequences for the 
way in which the concept of statistical entropy has developed. (See the entry 
Maxwell’s Demon in the Annotated Further Reading.) 

Maxwell’s conceit was to imagine a small, “neat fingered” being, later called 
a demon and much later called Maxwell’s demon , who attended a door between 
two chambers of air. (See Figure 2.6.) The air in the two chambers initially has 
the same temperature and density, but even so its individual molecules move 
at various speeds - a phenomenon first discovered by Maxwell. The demon’s 
job is to open the door in order, say, to allow the fastest moving particles in 
the right chamber to move to the left chamber and to allow the slowest mov- 
ing particles in the left chamber to move to the right chamber. In this way, the 
demon could cause the air in the left chamber to become hotter than the air in 
the right chamber while keeping their densities the same. If we consider the 
two chambers, the air they contain, and the demon as an isolated system, this 
process directly violates Clausius’s version of the second law assuming the 
entropy of the demon does not change. Admittedly, the latter is a significant 
assumption. 

Maxwell claimed that his demon-inhabited thought experiment demon- 
strated that the second law of thermodynamics “has only statistical certainty,” 
that the second law can at times be violated, and that the statistical entropy 
of an isolated system can momentarily decrease. For if a demon can cause 
the entropy of an isolated system to decrease, possibly an inanimate automa- 
ton like a spring-loaded, one-way door can do the same. Indeed, subsequent 
analyses and numerical simulations of many cleverly constructed, inanimate 
“demons” have shown that the statistical entropy of an isolated system may 
fluctuate - but only fluctuate. And the larger the system, the smaller is the rel- 
ative size of these fluctuations. 


2.6 Relative versus absolute entropy 
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Figure 2.7 When an internal constraint of an isolated system is relaxed, the sys- 
tem entropy (solid line) approaches and fluctuates close to and just under its max- 
imally probable, equilibrium value (dotted line). 


The entropy of a non-equilibrium system 


The machinations of Maxwell’s demon suggest a broader interpretation of the 
relative entropy according to which the multiplicity Q is that of an instanta- 
neous macrostate as a system evolves from an initial low entropy reference 
macrostate toward an equilibrium macrostate. This interpretation assumes that 
the system always occupies a macrostate at every instant of its evolution. In 
this case, S (£2) where 



(2.18) 


is the instantaneous entropy of a system, not necessarily in equihbrium, rela- 
tive to the entropy S (Q 0 ) of the system in a reference macrostate. Generally, 
the evolution of an isolated system is in the direction of increasing entropy. But 
around this general trend Maxwell’s demon teaches us there may be small fluc- 
tuations. As illustrated in Figure 2.7, the instantaneous macrostate multiplicity 
Q and hence the instantaneous entropy S (Q) fluctuate as the system occupies 
different, not quite optimal, macrostates. 


2.6 Relative versus absolute entropy 


The entropy relation S m c+ klnQ is perfectly adequate and, in fact, necessary 
for describing the entropy of a classical system. We will explore two such sys- 
tems in the next chapter: the ideal gas in Sections 3. 1-3. 5 and the ideal sohd in 
Section 3.6. Einstein and others recognized the general validity of S = c + k In Q. 
Nevertheless, one usually finds Max Planck’s more succinct result 


S = kl nO 


(2.19) 


in textbooks. 
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We have outlined the assumptions behind S = c + kin £2 in Section 2.4: the 
functionality S(£2) and the additivity and independence of isolated subsys- 
tems. In order to move from S = c + kin £2 to the simpler result S = /tin £2 we 
need a further piece of physics - one that does not undermine the role played 
by the constant c in enforcing the full content of additivity. The only generally 
applicable, empirically justifiable criterion that allows for this transition is the 
third law of thermodynamics. We will discuss this transition in Section 4.5. 

But late in the nineteenth century and in the early years of the twentieth cen- 
tury Planck was not aware of the third law of thermodynamics. Instead Planck, 
in addition to assuming the existence of a function S (□), additivity (2.1), and 
independence (2.3), implicitly, and without a priori justification, assumed that 
S (£2) is a universal function where here “universal” means a function only of 
the multiplicity £2 and universal constants. The universality of S (£2), additivity, 
and independence produce the result S = k In £2 + c where both k and c are uni- 
versal constants. If c is a universal constant then c is the same when applied to a 
system and its two isolated parts. In this case c A+B =c A +c B reduces to an equa- 
tion c = c + c whose only solution is c = 0. Thus Planck reduced S = k In £2 + c 
to S = kin £2. But in the process Planck eliminated his ability to impose the full 
content of a property he had originally assumed: additivity. 

Planck’s method actually worked for him because it anticipated future 
developments - in particular, the formulation and empirical justification of the 
third law of thermodynamics and the advent of quantum physics. Together 
these developments ensured that S (£2) can be both a universal function of mul- 
tiplicity £2 and an additive state variable. But some classical systems, including 
the ideal gas system, are neither third law compliant nor quantum mechan- 
ically conceived and for this reason are not well served by Planck’s equa- 
tion S = k In £2. For these classical systems we will use the relative entropy 
S = c + kin £2 in place of the absolute entropy S = kin £2. 


Problems 

2.1 Probabilities 

A container is divided into two compartments with volumes in the ratio 
V l /V 2 = 2. Volume V t contains 1000 N 2 molecules and volume V 2 contains 100 
0 2 molecules, (a) The interior wall separating the two volumes is punctured 
and the gases are allowed to achieve an equilibrium macrostate. What is the 
increase in the entropy of the two-gas system? (b) What is the probability that 
the gases will return to their original configuration? 


Problems 
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2.2 Playing card multiplicities 

(a) How many ways are there of drawing a 3 from a complete 52-card deck 
of playing cards? (b) How many ways are there of drawing a diamond from a 
complete 52-card deck? (c) How many ways are there of drawing a 3 from a 
complete deck of cards followed by a diamond drawn from a complete deck of 
cards? (d) What can you conclude about the dependence or the independence 
of the two events: “Draw a 3 from a complete deck of cards” and “Draw a dia- 
mond from a complete deck of cards?” 

2.3 Stirling’s approximation 

What is the percent difference between In 10! and Stirling’s approximate value 
10 In 10 - 10? How large must n be in order that the percent difference between 
Inn! and nlnn-n be less than 1%? 


2.4 The art of counting 

Because the multiplicity Q is the number of microstates available to a system 
in a given macrostate, we will, in the chapters that follow, often be presented 
with the problem of counting microstates. The following exercises introduce 
us to three different ways of counting microstates. You might try generating 
these formulas first by doing the counting when N and n are small numbers, (a) 
How many ways can N distinguishable particles he placed in n different boxes 
or cells? Find this number when IV = 50 and n = 100. (b) How many ways can 
N indistinguishable particles be placed in n different boxes or cells? Find this 
number when N = 50 and n = 100. (c) How many ways can N indistinguish- 
able particles be placed in n[> N] different boxes or cells when no more than 
one particle can occupy a box? Find this number when N = 50 and n = 100. 
(d) Which is larger: the answer to question (b) or the answer to question (c)? 
Explain why your answer makes sense. 

2.5 Rubber elasticity 

Consider the following simple, one-dimensional, statistical model of a rub- 
ber band. Suppose a length L of a band of rubber consists of N [» l] links 
where each link has length a. Some n + of these links extend in the positive 
x direction while n_ [= N — n + ] extend in the negative x direction. The orien- 
tation of the links contributes to the total length L = a [n + - n _ ) of the rubber 
band but does not change its energy. Therefore, a macrostate is defined by a 
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length L. (a) First, determine the multiplicity Q in terms of the length L. Use 
“N choose n + (b) Then determine the entropy in terms of L. (c) Justify the 
statement that the force F with which a length L of rubber pulls is related to the 
entropy S by F/T = ( dS/dL ) by inspecting the system’s fundamental constraint 
0 = TdS - FdL . Use this relation to determine an expression for the force F 
in terms of T and L. (d) Show that a linear, Hooke’s law relationship between 
F and L follows whenever L/ Na<gil. 


3 

Entropy of classical systems 


3.1 Ideal gas: volume dependence 

Boltzmann worked on the statistical mechanics of ideal gases for most of his 
professional life. We devote the next five sections to this task. Our ultimate 
goal is to construct an ideal gas entropy S {E,V,N) that is an extensive and 
so additive function of its extensive variables: internal energy E, volume V, 
and particle number N, and whose partial derivatives with respect to E and V 
produce the ideal gas equations of state: E = C V T and PV = NkT. Here we 
address the volume dependence and in the next section the conjoined volume 
and energy dependence of this entropy. 

We first employ a version of the relative entropy 

S(E,V,N)~c{E,N)+klnQ.(v) (3.1) 

that applies to a fluid system whose macrostate multiplicity is a function only 
of the system volume V. The positional microstates that compose the macro- 
states of this system are, in general, described by identifying the 3/V position 
variables: x v y v z v x 2 , y 2 , z 2 , ■ x N , y N , and z N , that locate the N particles of 
the system. 

In order to construct classical microstates that will help us determine the 
multiplicity Q we divide this 3/V-dimensional space into small regions or cells 
of size 8x l 8y 1 5z i 5x 2 ■ ■ ■ 5z N that are uniform and symmetric, that is, for which 
Sx, = 8y t = 8z x = Sx 2 = ■ ■ ■ - Sz N = 8s . Thus, each cell in this 31V-dimensional 
position space has volume 8s 3N = 8V N and each identifies a different classi- 
cal microstate. There are ( V/8V ) such cells. A single point representing the 
/V-particle system moves through this space and among these cells. 

Because there are ( v/8v) different cells accessible to the /V-particle system 
contained within volume V, the system multiplicity is given by il = (V / 8V) . 
Consequently, (3.1) becomes 
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S(E,V,N)=c(E,N) + Nk In 



(3.2) 


The fundamental relation of a simple fluid, dS = dE/T + PdV /T , implies that 
P/T = (dS/dV) EN and so, given (3.2), that 


PV = NkT, 


(3.3) 


which, of course, is the ideal gas pressure equation of state. Note that we have 
calculated the entropy function of an ideal gas only to the extent that from its 
volume derivative we can determine its pressure equation of state. Because 
we have not yet determined the complete dependence of the entropy on all its 
variables, we cannot yet derive all of the system’s equations of state. 


3.2 Ideal gas: volume and energy dependence 

In deriving an expression for the entropy S(E,V,N) that depends on both the 
internal energy E and the volume V of an /V-particle ideal gas we generalize the 
pattern of argument established in the previous section. The structure of the 
classical microstates available to this system is now more complex because it 
consists of both a particle position space and a particle velocity v or momen- 
tum p [= mv] space. Boltzmann realized that, because position and momen- 
tum coordinates play parallel roles in the Hamiltonian formulation of classical 
mechanics, microstates are best described in terms of position and momentum 
variables. 


Discretizing phase space 


The /V-particle, monatomic ideal gas system is described in terms of 3/V position 
jc, , y, , Z, , ...,z N and 3N momentum p xl , p yV p zV p zN coordinates. Together 
these coordinates define a 6/V-dimensiona] phase space through which a 
single point representing the /V-particle system moves. We discretize this 
6/V-dimensional phase space in order to define the classical microstates and 
determine the multiplicity of the macrostate. Accordingly, we divide the posi- 
tion space into uniform, symmetric cells of size 8x ] <Sy, & 8x 2 ■ ■ ■ 8z N = 8s 3N 
and the momentum space into uniform, symmetric cells of size 
8p x , 8p yl Sp z , Sp x2 ■ ■ ■ Sp zN = 5p 3N . We also adopt the following notation, 

8x 1 8y 1 8z l 5x 2 - 8z N 8p x] 8p y] 8p z] 8p x2 ■ ■ ■ Sp iN =(8s5pf N 


(3.4) 
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where H [= 8s 8p] is a quantity small enough to make the multiplicity of the 
system large compared to 1 but is otherwise arbitrary. The dimensions of H are 
those of action and also of angular momentum, that is, of energy multiplied by 
time with SI units of kg ■ m 2 /s . In Chapter 4 we will find that, according to the 
requirements of quantum mechanics, the arbitrary value of H is fixed by a uni- 
versal, determinate constant h - one we now call Planck’s constant. 

Identifying the cell in 6/V-dimensional phase space within which lie the 6/V 
coordinates of an ideal gas roughly identifies a classical microstate. We say 
“roughly” because one phase space cell necessarily encompasses many differ- 
ent points. The degree of roughness is determined by the size of H. The larger 
H, the rougher the description. 

The multiplicity 

Because the position and momentum coordinates of an ideal gas system are 
constrained in different ways, they do not play entirely parallel roles. For 
instance, the position coordinates must be consistent with locating the system 
particles within a volume V. For this reason the number of allowed position 
space cells, that is, the multiplicity associated with position space, is 



In contrast, the squares of the momentum coordinates must sum to the constant 
2mE so that 

pI,\ + Pli + pI\ + p \ 2 + P 2 y 2 + pI , 2 +••• + p 2 xN + p 2 yN + p\ N = 2mE. (3.6) 

where E is the system energy. Just as x 2 + y 2 + z 2 = R 2 defines a 3-dimen- 
sional sphere of radius R, the constraint (3.6) defines a 3/V-dimensional sphere 
in momentum space of radius \j2mE. The number of momentum space cells 
through which the system may move is the volume of the cells that he on the 
surface of this sphere divided by the volume 5p 3N of one such cell. And the 
volume of the cells that lie on the surface of the 3/V-dimensional sphere equals 
its surface area times a momentum space cell width 5p. 

This surface area of a 3/V-dimensional sphere can be calculated in the fol- 
lowing way. Just as the surface area of a 3-dimensional sphere AkR 2 is the 
derivative dldR of the volume of a 3-dimensional sphere AkR 3 /3 , the surface 
area of a 3/V-dimensional sphere A^ N (R) is the derivative dldR of the volume 
of a 3/V-dimensional sphere V 3N (r), that is, 


(3.7) 
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Furthermore, the volume V 3N (R) of a 3/V-dimensiona] sphere of radius R must 
be proportional to R 3N , that is, 

E w (fl) = C 3JV .R 3iV , (3.8) 


where C 3N 
In general, 


number independent of V and E. For example C 3 = 4 tt/3. 


C n = 


N 2 + 1]!' 


(3.9) 


Since we not concerned with cases for which n is a small integer, we replace 
(n/2 + 1)! in (3.9) with (n/2)\. And we apply Stirling’s approximation, 
lnn\~ n\nn — n 


~ In 


(3.10) 


or its equivalent 



(3.11) 


to expression (3.9). Therefore, 



(3.12) 


when n » 1. Using (3.12) in (3.8) and (3.7) we find that 



(3.13) 


Therefore, the volume of the momentum space through which the system may 
move is the volume [2eJi/3N) 3N/2 (3N)R 3Nl Sp of a one-cell wide shell of 
radius R = \[2mE . Consequently, the number of allowed momentum-space 
cells in this shell is 


(2en) 3NI2 (3N) 1 
where we have replaced R with \j2mE. 


.(2mEf N ~ 1)/2 l 

r s P ) 

l 31V J 1 

,Sp 3N ) 
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The multiplicity £1(e,V,N) of the /V-particle, monatomic, ideal gas with 
energy E and volume V is the product of the number of cells available in posi- 
tion space (3.5) and the number of cells available in momentum space (3.14). 
Therefore, 



where in the first step of this equation sequence we have, since N » 1 , dropped 
terms of order 1 compared to terms of order N. The terms dropped do not 
change the dimensionality of Q. We have, in the second step, also adopted the 
notation H = Ss 5p. 


The entropy of an ideal gas 

Accordingly, the entropy of an /V-particle, monatomic, ideal gas with energy E 
and volume V is 

5(£,T,A) = c(A) + Mn^|y ' j • (3.16) 

The two equations of state follow from the derivatives, (dS/dE) v [= \/T ] and 
{dS/dV) t . N [= P/T], implied by the fundamental relation dS = dE/T + PdV/T. 
Given the entropy (3.16) we find that, as expected, 

E = ~NkT (3.17) 

and 

PV = NkT. (3.18) 

(Of course, the energy of a system is known only within a constant. But if we 
replace E on the right hand side of (3.16) with E + E o and use \/T = (dS/dE) v 
and P/T = [dS/dV) EN , we again generate the familiar ideal gas equations of 
state (3.17) and (3.18).) 
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Summary 

We summarize the method employed in these last two sections as follows. We 
start with the relative entropy S [= c + k In Q] of an /V-particle ideal gas whose 
macrostate multiplicity Q depends upon the system internal energy E, volume 
V, and particle number N. Then we discretize the available 6/V-dimensional 
phase space and count the number of classical microstates through which the 
single point representing the system can move, that is, we determine the mul- 
tiplicity Q. This procedure produces the entropy S[E,V,N) relative to that of 
a reference macrostate. 

Both the multiplicity £2 and the entropy S determined in this way depend 
upon the size of the cells into which the phase space is divided. This cell 
size, in turn, depends upon an arbitrarily constructed quantity H [= 8s 8p\. 
Fortunately, none of the quantities that contain El can he measured, and none 
of the quantities that can be measured, for instance P, V, and T, depend on H. 
In classical statistical mechanics H is a mere methodological artifact with no 
empirical consequences. 


3.3 Imposing extensivity 

But more is required of the ideal gas entropy function (3.16). Any part of 
an ideal gas should itself he an ideal gas. In a word, the ideal gas should be 
extensive. For instance, if we divide the volume, the energy, and the number 
of ideal gas particles in half, we should create another ideal gas with half the 
entropy. In order that the entropy function (3.16) be extensive S ( E , V.N) must 
be a linear, homogeneous function of its extensive variables, that is, 

S(XE,XV,hN) = lS{E,V,N) (3.19) 


for arbitrary a. 

Recall that the purpose of the “constant” c[n) is to impose additivity, in this 
case extensivity, on the entropy. Together (3.16) and (3.19) require that 


S{XE,XV,XN) = XS(E,V,N), 
c(PiN) + UN J | + In \xv ^ j ) j | = lc(N) 


" ( 4mn 

{n) (W 


+ UN |2 + ln^ 

c(AA0 + UN In A = Ac (A) . 


)'! 


3.3 Imposing extensivity 


59 


Equation (3.20) has a family of solutions, 

c(N) = k(pN-NlnN), (3.21) 

where p is any constant real number. The particular choice p = 1 turns the 
entropy (3.16) into 

(3 - 22) 

an historically important function that has all the properties one can reasonably 
expect of the classical, monatomic, ideal gas entropy. As before the parameter 
H that determines the size of a cell in phase space is a methodological artifact 
with no empirical consequences. 

That the quantities p and H are arbitrary underscores what we already 
know: that the principles of a purely classical statistical mechanics based as 
they are on the first and second laws of thermodynamics do not invariably 
produce entropy functions that determine a unique value in terms of its exten- 
sive variables E, V, and N. We say “invariably” because imposing extensivity 
on blackbody radiation, as discussed below, does produce an absolute entropy 
with a unique value. But blackbody radiation is a particularly fundamental 
system that, for this reason, became an object of special interest in the late 
nineteenth century. 

Example 3.1 Extensivity of blackbody radiation 

Problem: Can the entropy of blackbody radiation be made extensive? 

Solution: According to (1.38) the entropy of blackbody radiation is 
described by 

s(£.y) = 4a '" 1 ' 3 "' £,,, + i , 

where a is the radiation constant and b is an arbitrary constant independent of 
the system’s thermodynamic variables E and V. Extensivity requires that 

s(ae,av) = as{e,v). 


that is, 




+ Ab, 


4 a l/4 (Av) m (Ae) 3 
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for arbitrary k. A vanishing b is the only value consistent with an arbitrary 
value of k. Therefore, b = 0 and 


S(E,V) = 


4 a m V m E 314 
3 


is the extensive entropy of blackbody radiation. 


3.4 Occupation numbers 

Although the ideas behind our derivation of the entropy of an ideal gas 
are simple ones, their application depends upon the unfamiliar geom- 
etry of 6/V-dimensiona] phase space. Boltzmann favored another derivation 
that employs a single 6-dimensional phase space common to each of the 
system’s N particles. The reduction of 6 N- to 6-dimensional phase space is 
possible whenever the particles that compose the system occupy their posi- 
tions in a common phase space independently of one another except for 
system-wide constraints such as those imposed on the total energy E and 
particle number N. 

Figure 3.1 illustrates Boltzmann’s idea for a positional phase space volume 
V common to all 10 particles of this model system. This positional phase space 
is divided into uniform cubes of size 5V that define a single positional micro- 
state of a single particle. Thus, there are V/SV such single-particle positional 
microstates. The classical microstate of this 10-particle system is identified by 
the particular arrangement of the 10 distinguishable particles in the common 
set of cells. 

More generally, Boltzmann employed special macrostates defined by 
the numbers of particles n. that occupy the jth cell [/= 1, 2, ...,/] of the 
6-dimensional, single-particle, phase space. The set of occupation numbers, 
n l , n 2 , . . . , Hj , defines a particular macrostate of the system. The occupation 
numbers that maximize the multiplicity Q of this macrostate also maximize 
the entropy S'(Q). This most probable macrostate defines an equilibrium 
macrostate. The use of special macrostates described by occupation num- 
bers might seem an unnecessary complication. But occupation numbers 
allow us to discover an important result that generally applies to many 
systems including both the ideal classical gas and the ideal classical solid 
that we explore in the following two sections. This important result is the 
Maxwell— Boltzmann distribution. 
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Figure 3.1 The volume V is divided into V/SV cells each of size 8V . One 
arrangement of 10 particles is shown. 

The Cartesian coordinates of a single-particle, 6-dimensional, phase space 
common to all particles are: x, y, z, p x , p , and p . We divide this phase space 
into 7 uniform, symmetric cells of size 

5x 5y 5z 5p x 5p y 5p z = (5s 5pf ^ ^ 

= H 3 

where 5s = 5x= 8y = 5z, 5p = 8p x = 5p r = 5p z , and H = 8s5p, and label 
these cells with index j = 1, 2, . . . , 7 . 

The system occupies a macrostate in which of the N distinguishable par- 
ticles n, are in the first cell, n 2 are in the second cell, . . ., and n, are in the 7th 
phase space cell. Imagine placing these N distinguishable particles in cells, one 
after another. We could do this by arranging these particles in a row, the first n, 
of which go in the first cell, the next n 2 of which go in the next cell and so on. 
Since there are N ! ways of arranging N distinguishable particles in a row, there 
are IV! ways of placing N distinguishable particles in the cells. However, reor- 
dering the particles within a cell does not generate a new microstate. Therefore, 
the number of arrangements that contribute to the multiplicity is N\ divided 
by the number of ways of ordering n t particles in the first cell, of ordering n 2 
particles in the second cell, . . ., and of ordering w 7 particles in the 7th cell, that 
is, by ! n 2 ! • • • rij !, for a total of N !/(«.! n 2 ! • • • n 7 !) distinct arrangements. This 
number iV !/(«j ! n 2 ! • ■ ■ n 2 l), clearly a generalization of the binomial coefficient, 
is called a multinomial coefficient. 

Therefore, the multiplicity of this occupation number macrostate is 

N' 

Q = (3.24) 

\n 2 ! ■■■n J ! 


and the corresponding relative entropy 
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= c + ^tVlnlV-X^n. \nn j J 


where in this step we employ Stirling’s approximation and in so doing assume 
that the size of a single phase space cell H 3 is large enough to make the occu- 
pation numbers n j » 1 for all j. 


Maximizing the entropy 

We maximize the entropy of this occupation number macrostate with respect to 
the occupation numbers n j where these latter are constrained by conservation 
of particle number 

N - Y' =i «. (3.26) 

and total energy 

£ = X- = 1 E i n r (3 - 27) 


Here e ; is the energy of a particle when it is in the jth cell of the single-particle 
phase space. For instance, when the system is an ideal gas the particle energy Ej is 
related to its particle momenta P X J , p yj , and p z j by e j = ( p 2 x j + p y , + p 2 z } j j2m . 
But other kinds of systems have other relations between particle energy e j and 
particle phase space coordinates p x p p y p z p x p y jt and z p 

We use standard methods of calculus to maximize the entropy with respect 
to the occupation numbers n p here considered as continuous variables, subject 
to the two constraints (3.26) and (3.27). These constraints are introduced into 
the maximization problem with two Lagrange multipliers a and fj. Thus we 
maximize the entropy of (3.25) by requiring that the partial derivatives of the 
constrained, normalized, relative entropy 

N In N - £ ^ In w . + ) + p[e - £ ^ n . e j j (3 ' 28) 

with respect to each n l vanish. The result is - In n. - 1 - a - j5e i = 0 which is 
solved by 




(3.29) 
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where the Lagrange multipliers a and fi are, in principle, determined by the 
constraints (3.26) and (3.27). Since the second partial derivatives of the con- 
strained relative entropy (3.29) with respect to the occupation number n j for 
all j are negative, this procedure locates a relative maximum and not only a 
stationary value. 


Single-particle partition function 

We eliminate the Lagrange multiplier a by summing both sides of (3.29) over 
all cells of the 6-dimensional, single-particle, phase space. This produces 

/V = e -('+«)^ / e ~t ^ ( 3 - 3 °) 


which, in turn, transforms Eq. (3.29) into the occupation number frequency 


n e -Pe< 

T 57 

or, equivalently and more compactly, into 


(3.31) 


N 


e -/fe, 

Zi ’ 


(3.32) 


where the quantity 

Z,=X>^ 


(3.33) 


is called the single-particle partition function and is so symbolized after the 
German equivalent Zustandssumme (state-sum). 

The single-particle partition function Z, and its generalizations are useful 
in statistical physics because they simplify many expressions and suggest an 
efficient order of calculation. For instance, given (3.32) and (3.33) the energy 
constraint (3.27) becomes 


E 

N 



InZ,. 

dp 1 


(3.34) 
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This result and (3.32) allow us to express the relative entropy (3.25) as 

S(E,V,N) = c(N) + kln[NlnN -^ P j=i nj(\nN -P£j -Inz)^ 

= c(N)+kln[NlnN-N]nN+PE + NlnZ l \ ( 3 - 35 ) 

= c(N) + k[PE+N]nZ l \, 

where here we have made explicit the dependence of the entropy S on the 
extensive fluid variables E, V, and N. 

Eliminating p 

Ordinarily we would use the energy constraint, (3.27) or (3.34), to eliminate 
the Lagrange multiplier /? in favor of the system energy E. But the algebra of 
this procedure is difficult if not impossible. Instead, we recall that any time we 
know how the entropy S of a fluid system depends upon its energy E we know 
the temperature of that system through l/T = (BS/dE) . To exploit this fact 
we use the relation (3.35) since it formally expresses the dependence of S on 
E. Then we find that 




UeJ, 

.» 1 V: J„ , 

■ffe) 



[ l ’>t> )} 


The latter, given the energy constraint (3.34), yields 



(3.36) 


(3.37) 


Although we could use (3.37) to completely eliminate the Lagrange multiplier 
fj from the entropy in favor of the temperature T, we often find it algebraically 
convenient to retain fi while remembering that P=l/kT. 


Maxwell-Boltzmann distribution 

The result P = l/kT turns the occupation number frequency (3.31) into an 
expression 
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N 


e~ El,kT 




(3.38) 


known as the Maxwell-Boltzmann distribution. Recall that in deriving the 
Maxwell-Boltzmann distribution (3.38) we assumed that the occupation 
numbers n are large compared to 1. When this is so, the ratio njN closely 
approximates the probability p i that any one particle will occupy the y'th cell 
of a single-particle, 6-dimensional, phase space. Accordingly, the independent 
particles of a system infrequently occupy cells with energies large relative to a 
characteristic thermal energy kT. 


Summary 

We summarize the occupation number method. The entropy of a system com- 
posed of N identical, yet distinguishable, particles is 

S(E,V,N) = c(N) + k[pE+NlnZ l ] (3.39) 

assuming that we can express fj [= 1 /kT ] and Z 1 in terms of the extensive vari- 
ables E, V, and N. Here 

E = -N — In Z, , (3.40) 

dp 1 

(3 ' 41) 

and 

P=— . (3.42) 

H kT 

Thus, the single-particle partition function (3.41) contains all the physics par- 
ticular to a given model while expressions (3.39) and (3.40) express algorithms 
for calculating entropy and energy. 

The sum in partition function (3.41) is over all the microstates available to 
a single particle. These microstates are defined by the different cells of uni- 
form size H 3 that compose the 6-dimensional phase space of a single parti- 
cle’s coordinates: x, y, z, p x , p , and p . Equations (3.39)-(3.42) apply to an 
/V-particle system whose particles share a common phase space, independently 
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occupy positions in that phase space, and do not interact except as necessary to 
maintain system-wide constraints on energy and particle number. 


3.5 Ideal classical gas 


We apply the occupation number method to an ideal classical gas by deter- 
mining its single-particle partition function Z r In doing so we assume, as did 
Boltzmann, that the size of a cell in phase space H 3 can be made large enough 
so that the number of particles in each cell n. varies only slightly between 
adjacent cells. Given this assumption the occupation numbers n. become con- 
tinuous functions of phase space coordinates, and we can replace the state-sum 
with an integral 




dxdydz.dp x dp y dp _ 

H 3 


(3.43) 


Consequently, 


lie ' dx\ dy\ dz\ dp x \ dp y j dp z e lie , (3.44) 


where 


£ = 


pI + p) + p- 

2m 


(3.45) 


is the energy of a particle of a monatomic ideal classical gas in terms of its 
phase space coordinates. And so 


^ .e Pe ’ -> -^\dx\d.y\dz\ dp x J dp y I dp z e ^ Pz *‘ ,y * Pz ) /2m 



V ( 2m7rV 

H 3 ( P ) 


(3.46) 


where in the last step we have used the identity | e x dx = \fn. Therefore, 
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V f 2 iwrV 
P J 


(3.47) 


is the single-particle partition function of a monatomic, ideal classical gas. 
This partition function contains all the physics of this model. 

The partition function (3.47) and the Eqs. (3.39)-(3.42) that summarize the 
occupation number method imply that 


V f 2mn\ 

H 3 { P ) 


(3.48) 


Furthermore, the entropy of the classical ideal gas is given by 
S(E,V,N) = c(N) + k[l3E+N]nZ 1 } 


= c(w)+M || + i„^|) 


Note that (3.49) recovers the multiplicity and entropy found in Section 3.2 
by discretizing the 6/V-dimensional phase space of the ideal gas system. The 
partial derivatives of the entropy function (3.49), ( dS/dE) yN [= 1/7’] and 
( dS /dV) £N [= P/T\, yield the equations of state: E = 3NKT/2 and PV = NkT. 


3.6 Ideal classical solid 

An ideal classical solid is one whose A particles oscillate independently in sim- 
ple harmonic motion with a common frequency v D around a crystalline array of 
fixed points as illustrated in Figure 3.2. Although these particles do not occupy 
the same positional space and, therefore, do not occupy the same phase space, 
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Figure 3.2. The classical ideal solid is composed of atoms or molecules that 
oscillate with simple harmonic motion (same frequency, various amplitudes and 
phases) around a crystalline array of fixed centers. Although only left-right motion 
is indicated here, the model includes motion in all three directions. 

they do occupy structurally identical phase spaces. For this reason we can use 
occupations numbers to determine the entropy of the classical ideal solid. 

Each particle has three position coordinates, x, y, and z, and three momen- 
tum coordinates, p x , p , and p . The range of the position coordinates encom- 
passes atomic or molecular sized regions of oscillation rather than the volume 
V occupied by the entire solid. Also, the energy e of each particle is the sum of 
its kinetic and potential energies, 



( 3 . 50 ) 


where co o [= 2?rv 0 ] is the oscillation frequency in radians per second, v 0 is the 
oscillation frequency in hertz, and mco 2 a is the elastic constant. All quantities 
are the same for all particles and for oscillations in all directions. 

Again we replace the state-sum in the partition function 



with an integral so that 


^ ] dx\ dy] dz I dp, ] dp y \ dp z . 


( 3 . 51 ) 


Consequently, 


^j e ^ — pJ dx\ dy\ dz\ dp x \dp y \ dp z e ^ 
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(3.52) 


and the single-particle partition function is given by 



Note that in the second step of the equation sequence (3.52) we extend the spa- 
tial integration over all space rather than over the system volume V or, more 
appropriately, over the relatively small volume of a typical sphere of oscil- 
lation. This tactic introduces only a small error, because the integrand of the 
spatial integrations, exp ^-/3m(0 2 o {^x 1 + y 1 +z 2 j^2j , becomes extremely small 
when the distance \jx 2 +y 2 +z 2 from the oscillation center is significantly 
larger than ^2 /({Jmafc') . For this reason, ^2 j^pma^ may he taken as the 

typical radius of the sphere of oscillation. This approximation of the spatial 
integrations also explains why the single-particle partition function (3.53) of 
the classical ideal solid is independent of system volume V. 

Given (3.53) the energy expression E = -Nd In Z l /d/3 reduces to E = 3 IV//?, 
that is, to the energy equation of state 

E = 3 NkT. (3.54) 

Consequently, the heat capacity C [= (d£/dT)J implied by (3.54) is given by 
= 3 Nk, (3.55) 

a result that reproduces the law of Dulong and Petit according to which the 
molar specific heat capacity C/n of a solid is always close to the universal 
constant 3 R [- 25 j/(K ■ mole) - 6.0 calorie/(K ■ mole)]. 


Equipartition theorem 

The similarity of the energy equations of state of an /V-particle, monatomic, ideal 
classical gas, E = 3NkT/2, and of an /V-particle, ideal classical solid, E = 3NkT, 
derives from their similar expressions for particle energy e, respectively, 
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£ = [pI + P 2 y + P 2 ,)/2m and E = [p 2 x + p 2 + p 2 z ^/2m + moor[x 2 + y 2 +z 2 )/2. 
Apparently, when the phase space integrations can be extended over all phase 
space, each term in the particle energy quadratic in a phase space coordinate 
contributes NkT/2 to the internal energy of a system ofN independent particles - 
a statement known as the equipartition theorem. When the equipartition theo- 
rem applies, a system’s internal energy is shared equally among its degrees of 
freedom. Because the energy of each particle of an /V-particle, monatomic, ideal 
classical gas contains three terms quadratic in the momenta, its internal energy 
Eis3xNxkT/2. Because each oscillator of an ideal classical solid contains six 
terms quadratic in its coordinates, its internal energy E \s6xNxkT /2. 

The entropy of an ideal classical solid 

Finally, we produce an expression for the entropy of an ideal classical sohd in 
terms of its macrostate internal energy E and particle number N. According to 
the occupation number method, S = c(N) + k(pE + NlnZ l ) where fl = 1 1 kT, 
and, according to our model of the classical, ideal solid E = 3 NkT and 
Z l = l/(pHv 0 ) 3 . Together these yield 



(3.56) 


Because this entropy does not depend upon the volume V, the pressure of 
an classical ideal solid vanishes - an unrealistic consequence of this overly 
simple model. 


Example 3.2 Average kinetic energy 


Problem: What is the average kinetic energy of a particle of classical, mon- 
atomic, ideal gas? And what is the root mean squared speed of a particle of 
this gas? 

Solution: Since all the energy of a classical, monatomic, ideal gas is con- 
tained in the kinetic energy of its particles, the answers to these questions fol- 
low almost immediately from the monatomic, ideal classical gas equation of 
state E=3 NkT/2. The average kinetic energy per particle, E/N=3kT/2, is a func- 
tion of the thermodynamic temperature T alone. This proportionality between 
the average kinetic energy and the thermodynamic temperature obtains for sys- 
tems that observe the equipartition theorem. 

If m is the mass of each gas particle, then the root mean squared velocity of 

the particles v 2 ) is determined by 
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m 



3kT 

2 


2 


Therefore, 



The hotter the gas, the faster its particles; the more massive the particle, the 
more slowly it moves. 


3.7 Boltzmann’s tomb 


It is perhaps surprising that the classical way of doing statistical mechanics 
has so little to do with mechanics per se, that is, with the science of motion. 
Nowhere do the methods of classical statistical mechanics require F = ma. 
Nowhere do they require solutions to equations of motion. Particles move, but 
move probabilistically according to certain rules: the fundamental postulate, 
conservation of energy, and conservation of particle number. That Boltzmann 
fashioned these ideas into a statistical method is, in retrospect, prescient. For, 
as we shall see, Boltzmann’s approach to entropy is almost perfectly congruent 
with the quantum physics that emerged after his death. 

Recall that the relation of relative entropy to macrostate multiplicity, 
S = c + fclnQ, follows from assuming that: (1) the entropy of an isolated 
system is a function S (il) of macrostate multiplicity O; (2) the entropies of 
isolated subsystems are additive; and (3) their macrostate multiplicities are sta- 
tistically independent. The most direct way of calculating the multiplicity O is, 
as in Sections 3.1 and 3.2, to discretize the phase space available to a system 
of given energy E, volume V, and number N of distinguishable particles and 
count the number of cells. 

In practice Boltzmann preferred the relatively indirect, yet powerful, occu- 
pation number method of determining the entropy of a system composed of N 
independent, distinguishable particles. According to this method we (1) dis- 
tribute the particles in a one-particle, 6-dimensional phase space that is divided 
into J cells; (2) define a macrostate in which the A particles of the system are 
partitioned into the numbers of particles, n v n 2 , ..., n p that occupy its J cells; 
(3) form the multiplicity Q of this macrostate N\/(n 1 \n 2 l...n J \); and (4) deter- 
mine the macrostate that maximizes its entropy S [= c + £lnQ] subject to the 
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constraints N = n l + n 2 + X + n } and E = n 1 e l + n 2 E 2 + ■ ■ ■ + n 2 e 7 . Both the direct 
and the occupation number methods produce a relative entropy rather than an 
absolute entropy. 

What then is one to make of the formula for absolute entropy, S = k log W, 
found on Boltzmann’s tomb in the Vienna Zentralfriedhof ( Central Cemetery)? 
Evidently, those who dedicated Boltzmann’s tomb in 1933 believed that 
S = k log W encapsulated Boltzmann’s contribution to physics. But this equa- 
tion appears nowhere in Boltzmann’s writings, originates with Max Planck, 
and incorporates physics of which Boltzmann could not have been aware. The 
last of these three statements may be controversial and, consequently, needs 
justification. 

But first, what does S = k log W mean? The convention that S stands for 
entropy had been established by Clausius in 1865 and k was Planck’s choice 
for what we now call Boltzmann’s constant (probably after the German word 
Konstant). But the letter W, taken from the German word Wahrscheinlichkeit 
for probability , is problematic. Certainly the word probability misleads. For 
probabilities are usually normalized to lie between 0 and 1 inclusive. And if 
0 < W < 1, the equation S = k log W leads to negative entropy - contrary to 
Clausius’s convention and our expectation. To be sure, Planck envisioned other 
ways of normalizing probabilities and called W not the probability but rather 
the thermodynamic probability. Others have referred to W as the thermody- 
namic or statistical weight. In actual practice, Boltzmann always used W as a 
part of the occupation number method and always replaced W by the multino- 
mial coefficient NMin^.n^.Xnfi. Thus W appears to be equivalent to what we 
call the multiplicity O of an occupation number macrostate of a system of N 
distinguishable particles. 

Given these identifications, Boltzmann’s epitaph, S = k log W, is equivalent 
to S = kin Q. And so we return to the question: What is the physics required to 
turn the equation describing the entropy of a classical system, S = c + klnQ, 
into the equation inscribed on Boltzmann’s tomb? As we shall see in Section 
4.5, this missing physics is the third law of thermodynamics - a law that 
allows one to adopt a conventional value for the entropy in the T —> 0 limit. 
But Walther Nemst (1884-1941) first articulated a version of the third law in 
1906 - too late for Boltzmann to exploit before his death in September of that 
year. And neither the ideal gas nor the ideal solid models that Boltzmann stud- 
ied are third law compliant. Evidently the equation S = k log W summarizes not 
only Boltzmann’s direct contribution to physics but also his service in laying a 
foundation upon which a new quantum statistical physics, then only emerging 
at the time of his death, could be built. 


Problems 
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3.1 Room temperature density 

Determine the number of particles in 1 cm 3 of ideal gas when T = 320 K and 
the pressure is 1 atmosphere. (Consult Appendix 1 : Physical constants and 
standard definitions.) 

3.2 Van der Waals equations of state 

The goal of much of statistical mechanics is to produce meaningful partition 
functions. But suppose someone gives you the ready-made single-particle 
partition function 



where a and b are constants that characterize a gas system and j3 = 1 / kT. Find 
(a) the corresponding energy equation of state and (b) the entropy function 
S(E, V, N). (Note that /? is a function of the extensive variables E, V, and N.) 
From the entropy function S(E,V,N) find (c) the energy equation of state 
again and (d) the pressure equation of state. 

3.3 Ideal gas of diatomic molecules 

The single-particle partition function of an ideal gas of diatomic molecules is 



where (5 = 1/ kT. Find (a) the corresponding energy equation of state and the 
pressure equations of state, (b) Use the equipartition theorem to make a mean- 
ingful statement about the number of quadratic terms in the energy of the 
diatomic molecules in this gas. 

3.4 Mixing of ideal gases 

Two containers of ideal gas, one of N 2 and another of 0 2 molecules, have the 
same volume V, pressure P, and temperature T. A valve connecting the two 
containers is opened and the resulting mixture comes to equilibrium. Assume 
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the container walls are adiabatic. What is the resulting increase AS in the 
entropy of this two-gas system? Use Eq. (3.49). 

3.5 Extensivity of ideal solid 

Impose the extensivity condition SikE.kN) = XS(E,N) on the entropy of an 
ideal classical solid given in (3.56). (Note: In much the same way that there 
is a family of extensive entropies of an ideal classical gas, there is a family of 
extensive entropies of the ideal classical solid.) 

3.6 Room temperature speed of N 2 molecules 

Use the data given in Appendix I, Physical Constants and Standard Definitions, 
to determine the root mean squared speed in meters per second of an N 2 mol- 
ecule at room temperature (T = 300 K). The mass of a nitrogen molecule 
is 28.0 u. 
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4.1 Quantum conditions 

Max Planck (1858-1947) initiated the era of quantum physics on December 14, 
1900 at a meeting of the German Physical Society by announcing a derivation 
of the spectral energy density of blackbody radiation that was consistent with 
the most recent data. Planck’s intention had been to apply Maxwell’s equations 
and Boltzmann’s statistical mechanics to electromagnetic radiation within a 
cavity whose walls were at a uniform temperature T, that is, to equilibrium or 
blackbody radiation. But along the way something unexpected happened: the 
arbitrary quantity H that regulates the size of the phase space cells remained in 
expressions of quantities that could be measured. The data available to Planck 
suggested that H has a definite, fixed value, henceforth denoted h, that is close 
to the modem value, 

h = 6.63x10 34 (4.1) 

s 

of this universal constant now known as Planck’s constant. 

Historians of science still debate the extent to which Planck was aware in 
1900 that his derivation depended upon revolutionary ideas. However, by 1908 
Planck was well aware and by 1925 physical scientists understood that quan- 
tum physics includes the following conditions: (1) phase space cells have a 
definite size determined by Planck’s constant h\ (2) the energy, momenta, and 
other dynamical properties of an isolated system are quantized; and (3) identi- 
cal particles are, for the purpose of determining multiplicity, indistinguishable 
from one another. 

While classically inexplicable, these conditions emerge naturally from 
the wave mechanics that succeeded the old quantum theory of 1900-1925. 
And while Clausius’s thermodynamic conception of entropy and Boltzmann’s 
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framework for relating the entropy of a system to its macrostate multiplicity 
survived the quantum revolution, the way we determine macrostate multiplic- 
ities has evolved. 

The classical way of doing statistical mechanics remains coherent and 
appropriate for working in, for instance, that regime in which the ideal gas 
equations of state are valid. But the classical way of doing statistical mechan- 
ics is also limited in its application and when applied outside of its appropriate 
regime makes predictions that are not in accord with measurements. 

A minimal application of the quantum conditions is simply a reinterpre- 
tation of the extensive entropy of an ideal gas (3.22) with Planck’s constant h 
replacing the arbitrary unit of action H. The result 


MW 


(4.2) 


is called the Sackur-Tetrode entropy after the two men who developed 
it independently in 1912. Recall that Boltzmann invented the quantity H 
[= 8s 8x\ in order to discretize phase space, but did not need to give H a def- 
inite value. Quantum physics, however, needs to and does assign H the def- 
inite value h l L = 6.63 x 10 34 m 2 kg/sj . Using Planck’s constant to discretize 
phase space produces in the Sackur-Tetrode equation (4.2) an expression 
for the absolute, extensive entropy of a quantized ideal, monatomic gas in 
the semi-classical regime which, as we shall see, is the low density, high 
temperature regime. 


4.2 Quantized harmonic oscillators 

The quantum harmonic oscillator exploits two quantum conditions: determi- 
nate phase space cells and quantized energy. 

Consider an array of N identical, simple harmonic oscillators that vibrate 
independently around fixed centers with frequency co o [= 2nv o ] where v o is in 
hertz. Although identical, these oscillators are distinguishable by virtue of the 
distinct positions of their centers of oscillation. The force constant of the linear 
force that returns the oscillators to their equilibrium positions is mar o where m 
is the oscillator mass. 

When these oscillators are one-dimensional, the state of each can be repre- 
sented as a point in two-dimensional, single-particle, x-p phase space as illus- 
trated in Figures 4.1a and 4.1b. 
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(a) 


(b) 


Figure 4.1 Oscillator positions in two-dimensional, single-particle, x-p phase 
space, (a) Oscillators randomly placed, (b) Oscillators placed on constant-energy 
ellipses with each ellipse separated by energy hv o . 


Oscillator energy quantized 


Figure 4.1a shows oscillators that seem randomly positioned in phase space. 
However, if we could observe their positions in sufficiently fine scale, we would 
see the oscillators naturally occupying positions on equal-energy ellipses as in 
Figure 4. lb For since the energy e of a single, one-dimensional oscillator is 
related to its position x and momentum p by 



oscillators that lie on the ellipse in phase space described by 



(4.4) 


have the same energy e. Furthermore, oscillators on the smallest ellipse 
have energy e 0 = Vzhv o , oscillators on the second smallest ellipse have 
energy e, = (I + V2 )hv o , oscillators on the third smallest ellipse have energy 
e 2 = (2 + Vi)hv o , and so on. Apparently, oscillator energy is quantized accord- 
ing to the rule 



(4.5) 


where j = 0, 1, 2, .... 


Oscillator phase space quantized 

One consequence of quantizing the oscillator energy in this way is that the area 
surrounding each ellipse in oscillator phase space is itself quantized. For since 
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the area of an ellipse is n times the product of its two radii, the area A. of the 
jth ellipse is 



where in the second step of this equation sequence we have used (4.5). Therefore, 
A J+1 -A. = h, (4.7) 

that is, the area between the j + 1 and j ellipses in the phase space of the quan- 
tized harmonic oscillator is Planck’s constant h. 

If the area of the smallest phase space cell of a one-dimensional system is h, 
what is its shape? In particular, why shouldn’t the shape of the smallest cells in 
oscillator phase space be that of elliptical annuli enclosing each equal-energy 
ellipse? After all, there can be no universal cell shape. For what would that cell 
shape be: rectangles, circles, wedges? Apparently each system has a cell shape 
appropriate to its dynamical structure. And, if the cells of oscillator phase space 
are elliptical annuli that enclose equal-energy ellipses, each such annulus has 
an area h. In this case, points that lie on one ellipse lie within one cell. 


Oscillator entropy 

Since the quantized oscillators are distinguishable one from another by vir- 
tue of their separate positions, the number of oscillators on the y'th ellipse is 
the occupation number n of the y'th cell and Boltzmann’s occupation number 
method, as summarized in Eqs. (3.39)-(3.42), applies. Furthermore, we need 
no longer assume that the occupation numbers n. are continuous functions of x 
and p. We can do the required sum exactly! This is fortunate since we can no 
longer assume that the single-particle phase space cells contain many particles. 
In particular, given that the microstate index j goes from 0 to oo, 


g -/*V„(/+ 1/2) 


_ g -^v„/2 


e -PhV„ + e -2phv € 


H 


1 +, 
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g-/9>v 0 /2 


(4.8) 


Since f5 = l/kT and, according to the occupation number method, 
rij/N = , the occupation number frequencies of a one-dimensional 

oscillator are given by 


(4.9) 


where j - 0,1,2,.... 

The energy of a system of N one-dimensional oscillators is 


E = -N — ]nZ, 

W 1 


= Nhv o 



(4.10) 


Solving (4.10) for hv o lkT in terms of the normalized system energy E/Nhv o , 
we find, after some algebra, that 


(4.11) 


This relation (4.11), the energy equation of state (4.10), and the partition func- 
tion (4.8) allow us to determine the entropy S ( E,N ) in terms of the system’s 
extensive thermodynamic variables. We find that 


S(E,N) = c(N) + k[/iE + N]nZ 1 ] 

= c(!V) + lvJf-^- + i 


— J — — —jin | — — — j 

1 Nhv„ 2 { Nhv, 2 J 


For the most efficient way of achieving this result, see Problem 4.1. 
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Three-dimensional oscillators 


Statistically 3N one-dimensional oscillators are equivalent to N three-dimen- 
sional oscillators when all have the same frequency v o . Therefore, the energy 
of N three-dimensional oscillators is given by (4.10) with N replaced by 3N, 
that is, by 


E = 3Nhv o 



(4.13) 


and the entropy of a system of N three-dimensional oscillators is given by 
(4. 12) with N replaced by 3N, that is, by 


S(E,N) = c(N) + 3Nk 



where we have redefined c(N). Albert Einstein (1879-1955) first derived and 
applied these results in 1907. 


Example 4.1 Correspondence principle 

Problem: What is the relation between the classical entropy and the quantum 
entropy of a system of N three-dimensional, linear, harmonic oscillators? 

Solution: The energy (4.13) and the entropy (4.14) of a solid composed of 
quantized oscillators should reduce to their classical counterparts in the appro- 
priate limit. In particular, the quantum entropy (4.14) should reduce to the 
classical entropy of an ideal solid (3.56) with H replaced by h, 

S ( £ ,a0«(a0 + 3m[im„[^|], 

of an ideal solid. Likewise the quantum energy (4.13) should reduce to the 
classical energy (3.54) of an ideal solid, 

E = 3NkT. 

Indeed, these reductions obtain in the limit of high energy per oscillator E/N 
relative to the unit hv o in which energy is quantized, that is, for E/N s> hv o or, 
equivalently, for high thermal energy kT relative to hv o , that is, for kT » /iv o . 
Such is one use of the correspondence principle according to which there is 
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always a limit, the so-called semi-classical limit, in which quantum expres- 
sions reduce to classical ones with the arbitrary H replaced with Planck’s 
constant h. 


4.3 Einstein solid 


Albert Einstein was probably the first, in 1907, to deliberately apply the quan- 
tum conditions to a physical model - to what has become known as the Einstein 
solid. An Einstein solid is a crystalline array of atoms or molecules each one 
of which oscillates simple harmonically in three dimensions with a common 
frequency v u . Einstein derived the energy (4.13) and the entropy (4.14) of a 
system of N such three-dimensional oscillators. 

Einstein was primarily concerned with the heat capacity C = dE/dT of a 
solid, that is, with 


(- hvJkT f e hvJkT 


C = 3nN A k- 


(4.15) 



where here n is the number of moles and N A is Avogadro’s number. The high 
temperature, semi-classical regime, kT ~»hv o , of this heat capacity recovers 
the law of Dulong and Petit, C = 3 nN A k [= 3nR\ . However, as k.Tlhv o -> 0 
the molar specific heat capacity C/n—> 0. In 1907 physical scientists were 
just becoming aware of molar specific heat capacities that drop below their 
Dulong-Petit value at relatively low temperatures. 

Einstein compared the dependence of C/n on T described by (4.15) to 
data available on the specific heat of diamond in one of only three graphical 
comparisons of theory and experiment he ever published. In doing so he used 
the quantity hv o /k, now known as the Einstein temperature, as a parameter 
with which to fit the data and found that hv o /k = 1325 K worked well for dia- 
mond. I have reproduced Einstein’s graph with his data converted to SI units 
in Figure 4.2. 

The Einstein solid is a rough model that in its details must be inaccu- 
rate. After all, because neighboring molecules exert the forces that restore a 
molecule to its equilibrium position, neighboring molecules do not oscillate 
independently as is assumed in deriving the Einstein solid. The resulting col- 
lective motion generates a whole spectrum of oscillations, not just a single 
frequency. Peter Debye (1884-1966) incorporated these collective oscilla- 
tions into a statistical model of solids that better fits specific heat data. But 
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Figure 4.2 Molar specific heat capacity C/n of diamond in joule/(kelvin mole) 
versus thermodynamic temperature T in kelvin. Filled circles: Measurements 
available to Einstein in 1907. Curve: Einstein’s prediction (4.15) with 
hvjk = 1325 K. 


it was Einstein’s 1907 paper that initiated the modem study of condensed 
matter physics. 


4.4 Phonons 

Because the energy of a simple harmonic oscillator is quantized in units of 
energy hv a where v a is the natural frequency of the oscillator, the energy of a 
system composed of such oscillators is also quantized. In particular, the nor- 
malized system energy in excess of its minimum, zero-temperature or ground 
state, value, that is, ( E -3Nhv o /2)/hv o is an integer. In the context of an 
Einstein solid hv a - sized bundles or quanta of energy are called phonons. Thus, 
energy flows from one place to another in the solid as phonons diffuse from 
one place to another. 

This way of thinking about the energy of an Einstein solid suggests an alter- 
native derivation of its entropy that uses each of three quantum conditions: (1) 
determinate phase space cells, (2) quantized energy, and (3) the quantum indis- 
tinguishahility of phonons. Imagine that each oscillator degree of freedom is a 
compartment that can hold any number of phonons. The Einstein solid consists 
of 3N such compartments. Together these compartments hold phonons. 
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Figure 4.3 An arrangement of seven phonons in five ordered compartments. The 
symbol O stands for a phonon and | for a divider between the compartments that hold 
the phonons. From left to right these compartments hold 2, 1, 4, 0 and 0 phonons. 


(. E-3Nhv a /2 ) 
hv o 


(4.16) 


The number of distinct ways of distributing n indistinguishable phonons 
among 3 N compartments is the system multiplicity fl. To motivate our cal- 
culation of the multiplicity let the symbol O stand for a phonon and | for a 
divider between compartments. One distribution of seven phonons into five 
compartments is shown in Figure 4.3. The order of compartments from left to 
right correlates with the order of oscillator sites and degrees of freedom. Here 
the first compartment contains two phonons, the second one phonon, the third 
four, and the fourth and fifth no phonons. 

Thus, the number of distinct ways n indistinguishable phonons may he put 
into 3 N compartments is (n + 3N — l)l/[n!(31V — l)lj - the binomial coeffi- 
cients again. This counting procedure naturally incorporates the conservation 
of certain quantities: the system energy E [= hv o (n + 3N/2)] and the number 
of oscillators N. Therefore, the multiplicity of a macrostate consisting of n 
phonons and N three-dimensional oscillators is 


(n + 3iV — l)! 

!(3AT-1)! ’ 


(4.17) 


and its entropy is 

S(E,N) = c(N) + k\n\ 


(n + 31V-l)! l 
n!(31V~l)! jj| 

= c {N) + k [(n + 3 N ) In (n + 3N) - n In n - (37V) In (3iV)] 
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= c(7V) + 3Mkl 

\3Nhv n 
= c(A)+3M:j 

(3A%v„ 


^ + 0lnU + ^l 

3 Nhv„ 2) [hv o 2 ) 



E 1 
3Nhv o + 2 




(4.18) 


where the first step in this sequence of equations follows from 3A » 1 and 
n » 1 and the second from (4.16). The fourth step redefines c(/V). This 
entropy (4.18) is the same as that (4.14) determined in Section 4.2 via 
occupation numbers. 

The method of phonons determines the entropy of a system directly in terms 
of the thermodynamic variables - here energy E and oscillator number N - 
without maximizing the constrained entropy as a function of occupation num- 
ber macrostate. When such direct methods work, they often work well. 


Indistinguishable particles and distinguishable places 

The direct method makes explicit the indistinguishability of identical phonons. 
After all, phonons are simply quanta of energy. Still one might object that 
phonons can become distinct from one another in much the same way that 
identical classical particles are so distinguished: by virtue of being confined 
to widely separated regions of phase space. But this could only be the case 
if there were many more oscillators than phonons, that is, TV » n, a regime 
that does, indeed, coincide with the classical limit. More generally, identical 
phonons are indistinguishable from each other. Thus, when two phonons are 
imagined to exchange positions a new microstate is not generated, just as when 
two dividers between the compartments that hold the phonons are exchanged a 
new microstate is not generated. 

However, the compartments themselves, that is, the oscillators, remain dis- 
tinguishable from each other - in Figure 4.3 by their very order. In general, the 
oscillators within a crystalline array are distinguished by the different places 
they occupy. Which identical objects are indistinguishable and which are not is 
a matter of context. In general those identical objects that permanently occupy 
distinct places in phase space are distinguishable from one another, while those 
identical objects that may occupy a common set of positions in phase space are 
indistinguishable. 


4.5 Third law 
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4.5 Third law 

We will continue to use the direct method of counting microstates and deter- 
mining the multiplicity O and entropy S of other systems, but first we exam- 
ine the consequences of applying the third law of thermodynamics. Recall the 
third law of thermodynamics: In the limit of vanishing thermodynamic temper- 
ature the entropy of a system is independent of its remaining thermodynamic 
variables. Max Planck observed that one is free to add to this empirical content 
a conventionally chosen value for this limiting entropy. 

Neither the monatomic, ideal classical gas nor the ideal classical solid 
observes the third law. After all, since the energy equation of state of each is such 
that E °= r , then E — » 0 as T — > 0. And given that the entropy S of each is a linear 
function of In E, the entropy of each diverges to negative infinity as T — > 0 . 

However, the Einstein solid does observe the third law. For, as T — » 0 the 
energy of the Einstein solid, given in (4.13), has the limit E — > 3Nhv a /2. 
Therefore, the entropy of the Einstein solid, given in (4.14) or (4.18), has the 
limit S(E) — > c[N). Since, in this case, we do not need the “constant” c(/V) in 
order to make the entropy extensive, we may, indeed, set c(/v) equal to zero, 
in order that the T — » 0 limit of the entropy becomes independent of all ther- 
modynamic variables. It may seem odd that a law of thermodynamics, usually 
thought of as a classical subject, depends so intimately on quantum physics for 
its observance. Yet, such is the case. 

The third law allows us to reinterpret and to simplify the relative entropy 
S [= c + k In Q] . First, observe that the relative entropy may be written 
in a form 

S(£2) = S(l) + klnQ (4.19) 

in which the constant c has been replaced by the entropy 5(1) of a macro- 
state composed of one microstate. Then, observe that Q — > £2.,._ 0 as T —> 0, and 
therefore that the entropy S — » S(Q r=0 ) as T —> 0. Applying this limit to the 
relative entropy (4.19) produces 

S (Q r _ 0 ) = 5 (l) + k In £2 r=0 . (4.20) 

Using (4.20) to eliminate S (l) from the relative entropy (4.19) yields 

5 (Q) = 5 (£V 0 ) + k filf 2 - • (4.21) 

According to the third law S (£2 r=0 ), and consequently must be indepen- 
dent of the thermodynamic variables that define a macrostate. This statement 
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incorporates the empirical content of the third law but not yet Planck’s 
convention. 


Planck’s convention 

Planck’s convention, that we are free to impose a convenient value on the 
ground state entropy S( Q r=0 ), allows us to go one step further. A commonly 
adopted convention is to choose 

S(Q r=0 ) = klnQ r=u (4.22) 

so that S (l) = 0 and the relative entropy (4.21) reduces to a version of the abso- 
lute or third law entropy 

S( Q) = klnQ (4.23) 

of an isolated system. Alternatively, we could choose S (£2 r=0 ) = 0 so that 
(4.21) reduces to S(Q) = kln(Q/Q r=0 ). The first choice, (4.22) resulting in 
(4.23), is traditional. We follow this tradition by adopting S = kin Q for the 
absolute entropy of a fully quantized and, subsequently, extensive, third law 
compliant system. 

In actual fact the two conventions are rarely distinct. Not only do fully quan- 
tized models of thermodynamic systems observe the third law of thermody- 
namics, but fully quantized models also usually have only one microstate at 
absolute zero, that is, usually Q T=g - 1. In this case, 5 , (£2 r=0 ) = 0 as well as 
S'(l) = 0, and the two conventions collapse into one. However, useful mod- 
els have been constructed of systems that are degenerate at T = 0, that is, for 
which £2 r=0 > 1. In this case, the absolute entropy S [= k In Q] of a system with 
a degenerate ground state does not vanish in the limit of vanishing thermody- 
namic temperature T. 


Constructing the entropy 

Nonetheless, many texts baldly assert S = klnQ as a definition of the statistical 
entropy of an isolated system. After all, this compact equation, first proposed by 
the great Max Planck, contains much physics. And that its equivalent appears 
on Boltzmann’s tomb adds to its appeal. But simply to assert S = klnQ makes 
sense only for fully quantized systems that are third law compliant. Applying 
S' = kin Q to a classical system such as the ideal gas invites paradox. 

Our strategy, beginning in Chapter 2, has been to require that the statisti- 
cal entropy be consistent with the first and second laws of thermodynamics 
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and to construct its form out of desired properties: dependence on multiplicity 
n, additivity, and the independence of isolated subsystems. This construction 
produces an entropy S [= S(l) + fc IntlJ relative to the entropy 5’(l) of a 
macrostate consisting of a single microstate and relative to an arbitrary phase 
space unit H. 

The quantum conditions require that Planck’s constant h replace the arbi- 
trary unit H. We also find that the third law of thermodynamics allows one to 
eliminate the entropy constant c [= S(l)]. Together these developments turn 
the relative entropy of classical thermodynamics into the absolute entropy 
of quantized, third law compliant systems. The option of using the absolute 
entropy S [= k In Q] arises only for quantized models. For only quantized mod- 
els remain accurate at low temperatures and, for this reason, only quantized 
models are third law compliant. 

Example 4.2 Shottky defect 

Problem: Determine the entropy S ( E,n ) of a crystal consisting of n lattice sites 
N of which are filled with molecules or ions and n - N of which are empty. 
Suppose the N particles occupying lattice sites in the crystal contribute no 
energy to the system while each of the n - N empty lattice sites adds energy e 
to the system. Express the entropy S as a function of temperature T and deter- 
mine whether or not this system is third law compliant. 

Solution: First we find the absolute entropy S = klnQ consistent with a 
given number of lattice sites n and particles N. The number of distinct ways of 
choosing n — N empty sites from among n possibilities is the binomial coeffi- 
cient, that is, “n choose n — N” 


[n — N)\N\’ 

where the number of empty sites n — N is related to the energy of the 
system E by 

E = {n-N)e. 


Therefore, the entropy in terms of E and n is 

S ( E ^ n ) = nlnn-(n-N)ln(n-N)-NlnN 



The relation 1 /T = ( dS/dE) n produces the equation of state 
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Using this result to eliminate the energy E in favor of the temperature T in the 
expression for entropy S produces 



Finally, as T —> 0, we see that e/kT —> °o and S — » 0. Therefore, this system is 
third law compliant. 


4.6 Paramagnetism 


A paramagnet is composed of nuclear, atomic, or molecular magnetic 
dipoles, essentially small magnets whose dipole moments tend to align with 
an externally applied magnetic field. Here we consider an especially simple 
kind of paramagnet composed of spin- 14 magnetic dipole moments fixed in 
a crystal lattice. The quantum mechanics of these spin- 14 dipoles requires 
that each dipole moment m B is either parallel or antiparallel to the applied 
field B o . 

When parallel to the applied field B o each magnetic dipole decreases the 
system energy by m B B o . Alternatively, when antiparallel each magnetic dipole 
adds m B B o to the system energy. If n + is the number of parallel and n the num- 
ber of antiparallel magnetic dipoles and all A lattice sites are filled with spin- 14 
paramagnets, then 


(4.24) 


and 


E = (n_ —n + )m B B o 
= (N -2n + )m B B o . 


(4.25) 


Consequently, 


N E 


2 2 m B B B 


(4.26a) 


and 
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N 

2 


E 


2 m B B o ' 


(4.26b) 


Thus, macrostates for which E = 0 are macrostates in which there are equal 
number of parallel and antiparallel dipoles. When the energy is near its mini- 
mum value E = -Nm B B o , most of the dipoles are parallel to the applied field 
so that n + ~ N and n_~ 0. And when the energy is near its maximum value 
E = Nm B B o , most of the dipoles are antiparallel to the apphed field so that 
n + ~ 0 and n_ ~ N. 


Entropy 

The absolute entropy S [= k In Q] of this system depends on the number of 
ways n + dipoles out of N can be chosen, that is, on N choose n + . Thus, 


Q = 


Nt 

n + \(N-n + )\ 


(4.27) 


L„ + !(A-u + )!j 

«&[lVlnfV-« + Inn. -(fV-n + )ln(iV-n + )J, 


where in the second step of this equation sequence we assume that N » 1 and 
( N - n + ) = n_ » 1 . Given (4.26a), 



Note that, according to (4.29), S(0,iV) = kN In 2. 


Energy equation of state 

Equation (4.29) tells us how the entropy S of the system depends upon its 
energy E. Thus, \/T = ( dS/dE) y N determines the energy equation of state 
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kT \ 2m B B uj ^ 

- — 

\2m B B o ) [l + E/(Nm B B o )\ 
Solving for E/(Nm B B o ) we find that 


E = Nm B B — — . 

B °y l+e 2» B Bj>T 


Note that when B o = 0, the energy of the system E = 0. Also, as the tem- 
perature approaches zero, T — > 0, the energy approaches its minimum value 
-Nm B B o with all paramagnet dipole moments parallel to the external field. 
Alternatively, very high temperatures, for which 2m B B o /kT <K 1 , will return 
the system to zero energy, E = 0, and effectively demagnetize the system. 

The mathematics of this model is, apart from an unimportant shift in the 
zero of energy, formally identical to that of a crystal with Shottky defects as 
described in Example 4.2. But the physics is significantly different. The energy 
of an empty lattice point, that is, of a Shottky defect, depends upon the extent 
to which the lattice points that surround the defect are filled. Therefore, the 
analysis of Example 4.2 applies only when the number of defects is small rel- 
ative to the number of lattice points. Not so for the model of paramagnetism of 
(4.24)-(4.31). After all, the energy of each dipole depends only upon an intrin- 
sic dipole moment and the strength of the applied field. 


4.7 Negative absolute temperature 

When the direction of the magnetic field applied to a paramagnet composed of 
spin- A dipoles is suddenly reversed, an interesting situation develops. Consider 
a system of paramagnets with more parallel than antiparallel paramagnets so 
that n + > n and, according to (4.25), E < 0. This situation arises naturally 
when a system of spin- 'A paramagnetic dipoles is cooled. Now imagine that 
the direction of the applied field B o is suddenly reversed. Then suddenly there 
are more antiparallel than parallel paramagnets so that n_ > n + and E> 0. This 
reversal is said to produce a population inversion of the magnetic dipoles. 

A system of population-inverted magnetic dipoles is in a region of parameter 
space in which an increase in the system energy E further increases the already 
relatively large number n of antiparallel dipoles, and, therefore, decreases the 
system entropy S. Recall that when we know how the entropy S of a system 
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nJN 

Figure 4.4 The normalized entropy S/kN of a system of spin-14 magnetic 
dipoles versus the fraction n + /N of parallel, low energy, dipoles as described 
by Eq. (4.32). 



Figure 4.5 The normalized inverse temperature 2m g B o /kT of a system of spin- 14 
magnetic dipoles versus the fraction n + / N of parallel dipoles as described by 
Eq. (4.33). 


depends upon its energy E, we know the absolute temperature of that system 
through 1 IT = ( dS/dE) y . And because, in this case, (dS/dE) v < 0, the abso- 
lute temperature of a population-inverted system of paramagnets is negative. 

Figures 4.4 and Figure 4.5 illustrate these facts. Shown in Figure 4.4 is 
a plot of normalized entropy S/kN versus the proportion n + /N of parallel 
dipoles from (4.28), that is, from 
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kN 



(4.32) 


and shown in Figure 4.5 is the normalized inverse temperature 2m B B o lkT 
versus n + /N, from (4.26a) and (4.31), that is, from 


2 m s B o 
kT 



(4.33) 


Reversing the direction of the applied magnetic field effectively shifts the sys- 
tem of spin- 1 /-! dipoles from the right- to the left-hand side of the entropy peak 
that lies at the point where n + /N = 1/2 and S/kN = In 2 in Figure 4.4 and from 
the right- to the left-hand side of Figure 4.5. 


Experimental realization 

Edward Purcell (1912-1997) and his colleague Robert Pound (1919-2010) 
were the first to create a population inversion among magnetic dipoles. Their 
chosen system, the nuclear spins of lithium ions in a lithium fluoride crys- 
tal, behave qualitatively as illustrated in Figures 4.4 and 4.5. Initially their 
crystal was in thermal equilibrium with the laboratory environment and in 
equilibrium with an applied magnetic field of 0.01 tesla. The applied field was 
then reversed within a millisecond, and the lithium nuclei quickly came to 
equilibrium among themselves and with the newly oriented applied field. The 
nuclei remained in this unusual population-inverted equilibrium for a period 
of minutes before again equilibrating with the laboratory environment. Purcell 
and Pound demonstrated that this system had a negative temperature by shin- 
ing monochromatic radiation with a frequency v [= 2 m B B o / h\ matched to 
the transition between nuclear spin states. That the incident radiation was 
enhanced through stimulated emission of the nuclei rather than absorbed 
showed that the nuclear spins were preferentially antiparallel to the applied 
magnetic field - just as expected if the system of nuclear spins had a negative 
temperature. These beautiful experiments, first performed in 1950, have since 
become routine. 

Negative temperatures can be achieved in any system composed of subsys- 
tems that (1) have a limited number of energy levels, (2) achieve equilibrium 
among themselves relatively quickly, and (3) equilibrate with their environment 
relatively slowly. For instance, the molecules that compose the active media in 
lasers and masers observe these three conditions. Such so-called “abnormal 
systems” seem exotic, but do not violate any law of thermodynamics. However, 
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most systems are “normal” in the sense that their entropy S increases monoton- 
ically with their internal energy E and, therefore, have positive temperatures. 


Problems 

4.1 Oscillator entropy 

The goal is to guide one to the efficient derivation of the entropy S of N one- 
dimensional, simple harmonic oscillators in terms of energy E and particle num- 
ber N as found in (4.12). Start with (4.08) and (4.11) and (a) express hv o lk.T 
and Zj in terms of the dimensionless quantity x = E/Nhv o . (b) Use these results 
and the relation S[E,V) = c (TV) + k [fiE + N In Z x ] to find the entropy function 
S[E,N) displayed in (4.12). 

4.2 Nitrogen molecules 

Diatomic molecules or dimers such as N 2 and 0 2 can be modeled as simple har- 
monic oscillators with a characteristic frequency of oscillation v o and energy 
levels e n — hv o (n + V 2 ) where n = 0,1,2,... A nitrogen dimer, for instance, is 
characterized by hv o = 0.3 eV. What is the proportion of room-temperature 
(300 K) nitrogen molecules in their first ( n = l) excited state? Note that 
k (300 K) = (1/40) eV. 

4.3 Correspondence principle 

Show that the Einstein solid, that is, a system of three-dimensional quantized, 
simple harmonic oscillators, observes the correspondence principle by show- 
ing (a) that its energy (4.10) reduces to the result 3NkT in the semi-classical 
li mit and (b) that its entropy (4.14) reduces to 3M[l + hi(£/3A%v o )J in the 
semi-classical limit. 


4.4 Two-level system 

A system consists of N distinguishable, independent particles each of which 
can exist in only two states: one with energy 0 and the other with energy e. 
Find (a) its single-particle partition function Z v (b) its energy E, and (c) its heat 
capacity C [= dE/dT], (d) Plot or sketch the dimensionless quantities C/kN 
versus e/kT from 0 to 3 and interpret the result. 
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4.5 Entropy maximum 

(a) Show that, according to (4.32), the relative frequency of a system of spin- 14 
magnetic dipoles parallel to an applied magnetic field n + / N that maximizes the 
entropy function S[n + ) is given by n + /N = 1/2. (b) Show that the maximum 
entropy of a system of spin- 14 magnetic dipoles is given by S = Nk\n2. 

4.6 Third law limit 

Show that the entropy of a system of spin- 14 paramagnets is third law 
compliant. 


4.7 Chemical potential 

An entropy that is a function S [E, V,n) of three extensive variables E, V, and 
N implies the existence of three equations of state. We have made use of the 
energy \/T =(dS/dE) y N and pressure P/T =(dS/dV) EN equations of state 
but not yet the equation of state containing the chemical potential p, that is, 
p/T = -f dS/dN ) E The chemical potential p is the energy added to the sys- 
tem per particle given that its volume and entropy are held constant, that is, 
p = (dE/dN) . . Derive the chemical potential equation of state for a system 
described by the Sackur-Tetrode entropy (4.2). 

4.8 Frequency of paramagnet orientations 

A crystal lattice contains spin- 14 paramagnets at each lattice site, (a) Derive 
expressions for the frequency of parallel paramagnet orientations n + / N and 
antiparallel paramagnet orientations n_/N in terms of the applied field B o and 
temperature T. (Here and in Problem 4.9 you may find it convenient to employ 
the notation x = 2 m B B o !kT.) (b) Using these expressions find the frequency 
of parallel and antiparallel orientations in the limits T — » °° and T — > 0. (c) 
Given that the value of a spin- 14 magnetic moment is given by m B = eh/4nm e 
(= 0.927 x 10“ 22 amp m 2 ) find the temperature at which 75% of the paramag- 
nets are aligned with an applied field of B 0 = 0.01 tesla. 

4.9 Heat capacity of a paramagnet 

(a) Find the heat capacity C [= dE/dT] of a paramagnet composed of N spin- 14 
dipoles with magnetic moment m B in an externally applied magnetic field B o . 

(b) What is the heat capacity C in the limits T -»-«>, and T — » 0? 
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4.10 Curie’s law 

The total magnetization M of a system of paramagnets is the sum of the individ- 
ual magnetic dipole moments. When the paramagnets are spin- Vi dipoles this 
sum is given by M = [n + - n) m B . (a) Find an expression for M in terms of N, 
B o , m B , and T. (b) Show that the magnetization M °c T 1 when 2 m B B o jkT <K 1 . 
The relation M = CB 0 /T, valid in this regime, is called Curie’s law. (c) Find 
the constant C in Curie’s law. 


5 
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5.1 Beyond the fundamental postulate 

According to the fundamental postulate, the microstates of an isolated sys- 
tem are equally probable. While the fundamental postulate is quite power- 
ful, it does not apply directly to systems that are not isolated. For instance, a 
very common non-isolated system is one in thermal equilibrium with its envi- 
ronment. When a system is not isolated we have no reason to believe that its 
microstates are equally probable. And when a system’s microstates are not 
equally probable, the entropy will no longer be proportional to the logarithm 
of the multiplicity. In fact, we have considered such a system before: a single 
particle of gas in thermal equilibrium with the other particles in the gas at tem- 
perature T. According to the Maxwell-Boltzmann distribution the probability 
that this particle will occupy microstate i with energy e ; is /z i which is 
certainly not the probability that it will occupy a different microstate with a 
different energy. In this section we will describe the entropy of an arbitrarily 
complex system whose microstates are not equally probable and give a more 
general derivation of the Maxwell-Boltzmann distribution. 

Ensembles 

We proceed by constructing, in our imagination, a collection of N indepen- 
dent copies of the system. The members of this collection are identical except 
that Hj (7 = 1,2,...) of the copies in the collection occupy microstate j with a 
frequency, 
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Figure 5.1 An ensemble of identical systems. Each system occupies a microstate 
indicated by the level on which the filled circle rests. Because the frequency with 
which a system occupies a microstate reflects a set of probabilities, the two middle 
microstates appear probabilistically favored. 


that reflects the probability p i that a copy will occupy its jth microstate. Of 
course, « and p i are constrained so that 




(5.2) 


and 


1 = 


(5.3) 


This collection of imaginary copies is called an ensemble. We owe the concept 
of an ensemble to J. Willard Gibbs (1839-1903). Its essential element is a set 
of microstate probabilities p,, p 2 , 


5.2 The Gibbs entropy formula 

Figure 5.1 illustrates a few members of an ensemble of identical systems in 
which each system occupies a microstate indicated by the level on which the 
filled circle rests. The systems shown in the figure appear to favor the middle 
two microstates. Since the members of an ensemble have been constructed in 
our imagination, we can tag or order them so that each is distinguishable from 
all others. For this reason we can apply the statistics of distinguishable objects 
to the members of an ensemble. 

Now consider the N distinguishable systems that compose the ensemble as 
a single composite system. The entropy of a macrostate of this N-subsystem, 
composite system is a function S(n t , n 2 , ...) of the occupation numbers 
Wp n 2 , ... that define its macrostate. Since the systems that compose the 
ensemble are distinguishable, each of the N\/[n t ! - - -) distinct arrange- 
ments in which n, of the N subsystems occupy microstate 1, n 2 of the N 
subsystems occupy microstate 2, and so on is a distinct microstate of this 
occupation number macrostate of the composite system we call an ensemble. 
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And since by construction the ensemble itself is isolated, each of these dis- 
tinct microstates is equally probable. Therefore, the entropy of this ensemble 
macrostate is given by 


S( n l , n 2 , ■■■n a )-S( 0, 0, ■••IV, •••0) = kln^— 

= /t{iVlnA-y .^lnnj 

= k {L j n j lnN j n i lnn j} 

= -kY n. In — 

^ 1 N 

= -kNT — In — 

N N 

= —kN y, Pi hi Pj , 


where we have used rij / A = Pj in the last step. The entropy S (0, 0, . . . N , . . . , 0) 
is that of the ensemble in a macrostate consisting of a single microstate, that is, 
a macrostate in which each of the N subsystems occupies the same microstate. 
If we make the entropy S (n L , n 2 , . . . , n n ) third law compliant, then 

5(0,0, •■■, N, -,0) = 0. (5.5) 


Furthermore, the additivity, more precisely the extensivity, of the entropy and 
n i = PjN assure us that 

n 2 , ■■■, Hu) - S{ Pl N, p 2 N, Pll N) 

= NS{p t , p 2 , ■••, Pa ) 

where S(p, n : , ■•■) is the entropy of the ensemble and S[ Pl , p 2 , ■■■) is the 
entropy of one of its identical subsystems. These requirements (5.5) and (5.6) 
transform (5.4) into an expression for the absolute entropy 

S{Pi, Pi, • Pn) = - k Y? j=l Pj ln Pj ( 5 - 7 ) 

of a system that occupies its microstates with unequal probabilities p v p 2 , 
..., p n . This result, sometimes called the Gibbs entropy formula, reduces to 
S = klnQ when p, = p 2 = ■■■ = p Q = l/£2. 


5.2 The Gibbs entropy formula 


99 


Random variables 


Note that, according to S = p j In p t , the dynamical variables of a non- 
isolated system must be random variables because the system realizes its 
microstates according to a set of probabilities Pj. In particular, the energy of a 
non-isolated system is a random variable because it realizes a range of values 
E U E % , ... according to a set of probabilities p v p 2 , .... Up until now we have 
considered only isolated systems that have a definite energy. The transition 
from systems described with so-called sure variables to systems described 
with random variables is significant. 

A random variable has an average value that is commonly called an expec- 
tation value. For instance, the expectation value of a non-isolated system’s 
energy, denoted (E), when that system realizes its microstate energy Ej with 
probability pj is defined by 


(*> IjPSr 


(5.8) 


Likewise, the pressure is also a random variable that assumes values P. 
[= -dZij/dv] with probabilities p } . Thus, 

I/jKM (5 ' 9) 


Expectation values are the observables of a thermodynamic system. When the 
system is isolated the expectation values (£} and (f) reduce to the sure variables 
E and P. Texts that make extensive use of probabilistic mechanics usually forgo 
special notation for expectation values and simply denote (p) and (p) w ith 
E and P - a possibly confusing abbreviation. The entropy S[p l , p 2 , ..., p n ) 

| = — , pj In Pj J is itself neither a random variable nor an expectation value 
but simply a function (5.7) of microstate probabilities. 


Eponymy 

Finally, we observe that the names Boltzmann entropy S [= k In £2] and Gibbs 
entropy formula S = — p , In p j are historically misleading. The history is 
complex but it is probably fair to say that Boltzmann discovered a continuous 
version of the Gibbs entropy formula while Planck introduced the Boltzmann 
entropy. This history reminds one of Stigler’s law of eponymy: No discovery is 
named after its original discoverer. 
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5.3 Canonical ensemble 


A canonical ensemble is one defined by probabilities p l ,p 2 ,... that characterize 
the microstates of a system in equilibrium with an environment at tempera- 
ture T. These probabilities maximize the Gibbs entropy S j^= —k^ p t In p ] J 
they determine given the normalization (5.3) and an expectation value (e) 
|^= ^jEjPj J . Maximizing the constrained entropy 


-J. I P^P l *a[l-J, i p j yi![{E)-l, jPj E I ] < 5 - 10 > 


with respect to p i produces 


In p. - \-a- [iE i = 0, 


(5.11) 


whose solution is the probability distribution 



(5.12) 


Since the second derivatives of (5.10) with respect to the microstate proba- 
bilities p t are negative, (5.12) identifies a relative maximum and not merely a 
stationary value of the constrained missing information. 

The Lagrange multipliers a and /? are, in general, determined by the prob- 
ability distribution (5.12) and the two constraints (5.3) and (5.8). Requiring 
that the probability distribution (5.12) satisfy the normalization condition 
(5.3) yields 




(5.13) 


Z’ 


where the system partition function 


z = Y e -^ 


(5.14) 


is a sum over the microstates of the system. Thus, the system partition function 
Z generalizes the single-particle partition function Z l used in Chapters 3 and 4. 
Given (5.13) the probability distribution (5.12) becomes 
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e -« 



Z ’ 


and the energy constraint (5.8) is equivalent to 


< £ > 


dlnZ 


(5.16) 


Together the probability distribution (5.15) and the Gibbs entropy formula 
S = In p. imply that 

S = k[p (E) + In Z], (5.17) 


Consequently, the derivative (dS/d (£)) [= l/T ] produces a result 


f = H< £ ) + - 


dp 


dP ] 

m) 


(5.18) 


that turns the probability distribution (5.15) into the so-called Boltzmann 
distribution 

e ~ E ^ 

Pt= (5-19) 
where Z = .e !1E ' and p = 1/kT. 

This derivation of the Boltzmann distribution (5.19) is formally identical to 
the derivation of the single-particle Maxwell-Boltzmann distribution in Section 
3.4, but the context is much broader. The microstate energies E. in (5.19) are 
those of any system however complex or simple and the sum in the partition 
function (5.14) is over all its microstates. If one first determines the partition 
function Z = ^ e l>E ‘ , then the energy equation of state (£') = -din Z/dp, 
the entropy S = fc[j3(£} + lnZ], and the Boltzmann distribution p j = e i>h ’ /z, 
where P=l/kT, all follow. The partition function is a flexible and straightfor- 
ward approach to statistical mechanics that avoids multiplicities and combina- 
toric formulas. 
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5.4 Partition functions 

The systems we consider in this book are composed of identical particles that 
occupy their single-particle microstates independently apart from system- wide 
constraints. In this case the energy E h of a particular system in microstate h is 
a simple sum, 

ti h = £ u + t' 2 , y + •"+£*,* > (5.22) 

of the energies, £ u , e 2j , ... e Nk of the N particles that form the system when 
in this microstate. Here, for instance, the subscript 2, k would indicate the 
energy e 2k of the second particle when in its kth single-particle microstate. 
The subscripts i,j, ...,k together define the system microstate h. Therefore, the 
system partition function is given by 

-I „e (5.23) 



where the sum is over all values of the N single-particle indices i, j, k that 
determine the system microstate. 

When in addition to their independence the identical particles that com- 
pose the system are distinguishable from one another, each realization of one 
particle’s single-particle microstate combines with each realization of all the 
other particles’ single-particle microstates to make a distinct /V-particle system 
microstate. Then the partition function sum (5.23) is further decomposed: 

z _ y g-Jfew 

-I, 2,-£, «-*<-*•••«* 

< 5 - 24 > 

- z /\ 


Given this result, those equations that encapsulate the partition function method 
applied to a complex system (5.14), (5.16), (5.17), and (5.19) reproduce sim- 
ilar equations in terms of the single-particle partition function as derived in 
Section 3.4. 
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Equations (5.14), (5.16), (5.17), and (5.19) also reduce to familiar results in 
the special case when all the microstates of the system are degenerate, that is, 
when all microstates have the same energy E. In this case 

z = y 

i (5.25) 

= Q e -’ m , 

where fl is the number of system microstates with E. The resulting entropy, 

S = k[/3E + lnZ\ 

= k\pE + ]nQ-pE\ (5.26) 

= JtlnQ, 

is, as expected, that of an isolated system, or alternatively, that characteristic of 
a system in a so-called microcanonical ensemble. 

We will not, in this Guide, advance the random variable view of statis- 
tical mechanics much further. But see Problems 5.1 and 5.2 and Sections 
8.4 and 8.7. 


5.5 Entropy metaphors 

The Boltzmann (5.26) and Gibbs entropy (5.7) formulas provide us with formal 
answers to the question “What is entropy?” But they also suggest appropriate 
metaphors that summarize the content of the entropy concept. For instance, the 
Boltzmann entropy S [= k In Q] suggests that entropy is spread in phase space. 
For an isolated system whose macrostate consists of many microstates, that is, 
one for which Q » 1 , probabilistically inhabits a correspondingly large vol- 
ume of phase space, and an isolated system that occupies only one microstate, 
so that Q = 1, is as confined in phase space as can be. 

According to the Gibbs formula the entropy is a function of the probabil- 
ities with which a system occupies its accessible microstates. This function 
is largest when the microstate probabilities are equiprobable, that is, when 
p, = p 2 = - and smallest when one microstate is certain, say, when p l = 1 and 
p 2 = p 3 = ■ ■ ■ = 0. Thus, an event whose possible outcomes are equally proba- 
ble is maximally uncertain and one whose possible outcomes have vanishing 
probability except for one with probability one is minimally uncertain. In this 
sense, the Gibbs entropy S J^= — , Pj In p j J is a measure of the uncertainty 
of a system’s microstate. Thus, the metaphor uncertainty also conveys the 
meaning of entropy. 
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Problems 

5.1 Canonical ensemble of Shottky defects 

The context is Example 4.2. The system is a lattice site within a crystalline 
array of lattice sites in thermal equilibrium with each other at temperature T. 
When occupied by an atom or molecule the lattice site has energy E 0 = 0 and 
when empty the lattice site has energy E 1 = e. (a) What is the probability p 0 
that the lattice site will be occupied and the probability p t that the lattice site 
will be unoccupied? (b) Use these probabilities to determine the entropy S 1 
|^= -*£ . p t In p t J of a single lattice site, (c) Show that the entropy of N lattice 
sites S [= NS t ] is consistent, with the entropy S of a crystal with N sites con- 
taining n Shottky defects as determined in Example 4.2. 

5.2 The Gibbs entropy formula 

Consider a system in equilibrium with its environment at temperature T. The 
probability that this system occupies microstate i is given by p l = e / Z 
where £ is the energy of the system when in microstate i and its partition 
function Z = ^ e ,SE ‘ . From this result and the Gibbs entropy formula derive 
Eq. (5.17) for the entropy S of the system and Eq. (5.16) for the expected 
energy (E) j^= ^ p : E t J of the system expressed in terms of the system 
partition function Z and the factor fi [= \/kT ]. 
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6.1 Symmetries and wave functions 

An important difference between the classical and quantum perspectives is 
their different criteria of distinguishability. Identical particles are classically 
distinguishable when separated in phase space. On the other hand, identical 
particles are always quantum mechanically indistinguishable for the purpose 
of counting distinct microstates. But these concepts and these distinctions do 
not tell the whole story of how we count the microstates and determine the 
multiplicity of a quantized system. 

There are actually two different ways of counting the accessible microstates 
of a quantized system of identical, and so indistinguishable, particles. While 
these two ways were discovered in the years 1924-1926 independently of 
Erwin Schrodinger’s (1887-1961) invention of wave mechanics in 1926, their 
most convincing explanation is in terms of particle wave functions. The follow- 
ing two paragraphs may be helpful to those familiar with the basic features of 
wave mechanics. 

A system of identical particles has, as one might expect, a probability den- 
sity that is symmetric under particle exchange, that is, the probability den- 
sity is invariant under the exchange of two identical particles. But here wave 
mechanics surprises the classical physicist. A system wave function may either 
keep the same sign or change signs under particle exchange. In particular, a 
system wave function may be either symmetric or antisymmetric under parti- 
cle exchange. 

For this reason identical particles come in two and only two kinds: bosons 
(for instance, phonons, photons, and helium-4 atoms) described by symmetric 
wave functions and fermions (for instance, electrons, protons, neutrons, and 
helium-3 atoms) described by antisymmetric wave functions. A consequence 
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of the different symmetry properties of bosons and fermions and of the fact that 
wave functions superpose is that the probability density of a system of bosons 
is non-zero where two bosons inhabit the same single-particle microstate and 
the probability density of a system of fermions must vanish where two fermi- 
ons are imagined to occupy the same single-particle microstate. Thus there are 
two kinds of identical particles: those that can occupy the same single-particle 
microstate (bosons) and those that cannot (fermions). 

The spin-statistics theorem and the Pauli principle 

The spin-statistics theorem, discovered by Wolfgang Pauli (1900-1958) in 
1940, builds upon these features of wave mechanics. According to the spin-sta- 
tistics theorem, all particles with even units of half-integer spin are bosons and 
all particles with odd units of half integer spin are fermions. Spin is intrinsic 
angular momentum. The unit of spin is the reduced Planck constant H [= h/2n] . 
Therefore, one unit of half-integer spin is K/2. Pauli had in 1924 proposed 
the rule that electrons within an atom cannot occupy the same single-parti- 
cle microstate - the so-called Pauli principle - without at the time realizing 
that this behavior was characteristic of a larger class of particles we now call 
fermions. 

We summarize. The only important distinction for the statistical mechanics 
of quantized systems is the distinction between bosons, which can share the 
same single-particle microstate, and fermions, which cannot. Bosons are said 
to obey Bose-Einstein statistics, named after the two men, Satyendra Nath 
Bose (1894-1974) and Albert Einstein, who in 1924 independently discovered 
(Bose) and applied (Bose and Einstein) this way of counting distinct micro- 
states. Fermions obey Fermi-Dirac statistics named after the two men, Enrico 
Fermi (1901-1954) and P. A. M. Dirac (1902-1984), who in 1926 indepen- 
dently discovered and applied this way of counting distinct microstates. In the 
previous chapter we investigated systems of bosons (phonons) and continue 
to do so in the next (photons and bosons with non-zero rest mass). Here we 
explore the statistical mechanics of fermions. 


6.2 Intrinsic semiconductors 

The single-particle microstates available to the electrons of a material in its 
condensed phase are of two kinds: bound or valence band states and conduc- 
tion band states. Since electrons are fermions, at most one electron at a time 
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Figure 6.1 Eight valence band microstates and eight conduction band single-par- 
ticle microstates. Three electrons have been excited from the valence band to the 
conduction band. 


can occupy a bound or conduction band single-particle microstate. Typically 
the energies of the bound microstates crowd together, as do the energies of the 
conduction band microstates, while a bandgap of up to several electronvolts 
may separate these two groups or bands of single-particle microstate energies. 
The exact size of the bandgap determines the conductivity of a material. The 
bandgap is relatively large for insulators, relatively small for semiconductors, 
and nonexistent for conductors. 

The most common solid-phase semiconducting material is crystalline sil- 
icon. An intrinsic semiconductor is a crystal with no defects or impurities. 
When lattice vibrations excite a valence electron into the conduction band of 
an intrinsic semiconductor, both the vacancy or hole left in the valence band 
and the newly formed conduction electron are free to move under the influence 
of an applied electric field. Holes and electrons contribute in equal measure to 
the conductivity of an intrinsic semiconductor. 

Figure 6.1 illustrates the simplified model we consider. The numbers of 
valence and conduction band single-particle microstates are equal as are the 
number of holes and conduction electrons. In this illustration the valence 
and conduction band single-particle microstates have been made degenerate 
with one energy level for each band. The valence and conducting bands of a 
more complete model would consist of closely spaced non-degenerate energy 
levels. This model captures one important feature of intrinsic semiconductors - 
bandgap energy - at the cost of ignoring other features. 

Entropy 

The arrangement of holes in the valence band and the arrangement of elec- 
trons in the conduction band both contribute to the multiplicity of an intrinsic 
semiconductor. Suppose the semiconductor has N electrons, N valence band 
single-particle microstates, and N conduction band single-particle microstates. 
Of these A electrons, n occupy the conduction band and, because the number of 
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electrons is conserved, N-n are left to occupy the valence hand. Also suppose 
the two bands are separated by bandgap energy e. 

Because electrons are fermions, no more than one electron can occupy 
each single-particle microstate. When n electrons are excited from the valence 
band to the conduction band, the A valence band, single-particle microstates 
divide into two groups: one group of n microstates that are unoccupied holes 
and another group of N-n microstates that are occupied by one electron. 
Therefore, the binomial coefficient A !/[«!( A - m)!J gives the number of, 
equally probable, arrangements of the valence band subsystem. Similarly, the 
A conduction band microstates divide into two exclusive groups: n are occu- 
pied by a single electron and A - n are unoccupied. Therefore, the number 
of equally probable arrangements of the conduction band subsystem is also 
A \j\ji !(/V -«)!_]• Because each valence band subsystem arrangement can be 
paired with each conduction band subsystem arrangement to form a single 
system microstate, the multiplicity of the intrinsic semiconductor system is 


Q = 



(6.1) 


and therefore its entropy is given by 

= 2k [ A In A - n In n - ( A - n) In ( A - n)] (6.2) 



where we have assumed that N - n » 1 and n » 1 . 

Since the energy of the system, apart from an inconsequential constant, is 
E = ne, (6.3) 

where e is the bandgap energy and, of course, n is the number of conduction 
band electrons, the entropy S of the intrinsic semiconductor as a function of 
system energy E is given by 



S(£, A) = 2A:/V 


(6.4) 
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The energy equation of state 

We may now determine the energy equation of state from (6.4) and 
\/T = (dS/dE) N . In this way, we find that 



Solving (6.5) for the energy E produces the energy equation of state 


That the frequency of occupied conduction band, single-particle microstates 
is given by 


(6.7) 


follows immediately from (6.3) and (6.6). As one might expect, in the limit 
of vanishingly small absolute temperature the frequency of conduction band 
electrons vanishes, that is, n/N—> 0 as e/2kT — > °o. In the limit of indefi- 
nitely large absolute temperature the electrons are shared equally between 
valence and conduction bands, that is, n/N — > 1/2 as e/2 kT — » 0, also as one 
might expect. 

Typical semiconductor bandgap energies are on the order of a few electron- 
volts (1 eV=l .60 x 10“ 19 joules). When, for instance, the bandgap energy s = 2 eV, 
the frequency of occupied conduction band single-particle microstates n/N, 
at room temperature (T = 300 K and kT = 0.026 eV), is given by 


N 1 + g 1 / 0026 

1 ( 6 . 8 ) 
1 + e 38 ' 5 
= 1.9xl0~ 17 . 


Thus, typically, an incredibly small fraction of the electrons and their holes con- 
tribute to the conductivity of an intrinsic semiconductor at room temperature. 
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Because the bandgap energy e and the absolute temperature T enter into the 
argument of the exponent in expression (6.7), the frequency of occupied con- 
duction band single-particle microstates n/N is quite sensitive to small changes 
in e and T. 


6.3 Ideal Fermi gas 

We have used two different methods to derive the entropy of an ideal gas and 
the equations of state that follow from that entropy in Chapter 3: (1) the direct 
method in which one counts the number of microstates accessible to a gas of 
N particles that share energy E and occupy volume V, and (2) the occupation 
number method in which one maximizes the number of ways of distributing 
particles among the cells that compose a single-particle phase space while con- 
serving N and E. These methods as applied in Chapter 3 share at least two 
distinctly classical assumptions: (1) identical particles are considered distin- 
guishable and (2) the entropy is given by S = c + k In Q where the “constant” c 
is chosen to make the entropy S {E, V, N) an extensive function of its variables. 
These methods produce an extensive entropy and the familiar classical, ideal 
gas equations of state. 

We are now in a position to revisit the ideal gas from the quantum per- 
spective. We implement this perspective in four ways: (1) we quantize phase 
space with Planck’s constant h\ (2) we drop the classical assumption that 
identical particles are distinguishable; (3) we use the third law compliant 
absolute entropy S = klnQ; and (4) we assume in this chapter that the iden- 
tical particles are fermions and in the next that they are bosons. Given these 
assumptions we will find that the entropies of the ideal Fermi and ideal 
Bose gases are extensive functions S(E,V,N) that in one limit reproduce 
the Sackur-Tetrode entropy of an ideal classical gas while more generally 
retaining quantum features. 

Because we retain the idealization that there are no interparticle forces, we 
continue to speak of an ideal gas: in this chapter of an ideal Fermi gas and in 
the next of an ideal Bose gas. We also simplify our investigation by assum- 
ing that each particle of the gas possesses the average energy E/N of the 
gas system. This average energy approximation produces results that, while 
approximate, describe important features of the ideal Fermi and ideal Bose 
gases. Other approaches to the ideal quantum gases that incorporate more 
physics can be found in the books listed in the Annotated Further Reading in 
Appendix VI. 
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The multiplicity of distinguishable particles 

First a review. Recall from equation (3.15) that the multiplicity fl of an ideal 
gas composed of N distinguishable particles that occupy volume V and share 
energy E is given by 



Recall also that the symbol e stands, in this context, for the base of the natu- 
ral logarithm. The factor in the brackets on the right hand side of (6.9) can be 
expressed as 

a(E/ N ,v, (6.10) 

in which case (6.9) becomes 

□ (£,y,Ar) = [£2(£/A',V,l)] A '. (6.11) 

If Q.(e,V,N) is the multiplicity of an ideal gas composed of N distin- 
guishable particles, possessing energy E, and occupying volume V, then 
£l(E/N ,V,l) must be the multiplicity of a single particle of ideal gas, pos- 
sessing energy E/N, and occupying volume V when that particle is part of 
a larger system of N distinguishable particles. According to this interpreta- 
tion a single particle of ideal gas may occupy Ll(E/N ,V,\) equally probable 
microstates, a system of two distinguishable ideal gas particles may occupy 
[Q(E/iV, V,l)] equally probable microstates, and so on until a system of N 
distinguishable particles of ideal gas may occupy [£2(£/lV,V,l)] equally 
probable microstates. 


The multiplicity of an ideal Fermi gas 

The multiplicity Q.{E/N ,V, l) of a single particle of ideal gas with energy 
E/N is, of course, the number of cells that can be occupied by that particle. 
Furthermore, the number of cells that can be occupied by a single particle of 
ideal gas must be independent of whether that particle is itself distinguishable 
or indistinguishable from the other particles that compose the gas system and 
if indistinguishable whether fermion or boson. This crucial insight allows us to 
proceed in the following way. 
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_Q_ 


_o_ _e_ 

. _a_ _e_ 

Figure 6.2 The six distinct ways of filling four cells with two fermions. 


First, we establish the following notation. Let n stand for the number of cells 
that can be occupied by a particle of ideal gas with energy E/N, that is, let 
n = Q(E/N,V, 1) 

- ^ 47rew J /2 (6 ' 13) 

where in (6.13) we have used (6.10). Second, we count the number of dis- 
tinct ways of filling n cells with N [< n] fermions while leaving n - N of 
them empty. 

The problem of filling n cells with N [< n\ identical fermions has already 
been solved and exploited in Section 6.2 even if there the symbols n and N 
had different meanings. As an example consider the number of distinct ways 
of filling four cells with two fermions as illustrated in Figure 6.2. There are 
4 1/[(4 - 2) ! 2 !] [=6] distinct ways. In general there are “n choose N” or 


(n-N)\N\ 


(6.14) 


distinct ways of filling n cells with N fermions. 

The multiplicity described by (6.13) and (6.14) is the result of the aver- 
age energy approximation. According to this approximation each gas particle 
possesses the same energy: the average energy E/N of the particles in the sys- 
tem. That the average energy approximation produces the exact multiplicity n N 
when identical particles are considered distinguishable encourages us to use 
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this approximation for the quantum ideal gases when identical particles are 
indistinguishable. 


The entropy 

Given (6.14) the entropy of an ideal Fermi gas is 



(6.15) 


where in the first step we have assumed that N » 1 and n - N s> 1 in order 
to use Stirling’s approximation. Also, recall that the number of single-particle 
cells is given by (6. 1 3) and that the number of fermions N can be no larger than 
the number of cells n. Thus, 



(6.16) 


describes the domain of the average occupation number per cell N/n or, more 
briefly, the occupancy of a system of identical fermions. 

This domain (6.16) reminds us that the ideal Fermi gas may not always 
realize one of the conditions, n - N » 1 , assumed in deriving the entropy 
(6.15). In fact, at very low temperatures the occupancy N/n of an ideal Fermi 
gas approaches 1. For this reason, the assumption that n-N» 1 limits the 
model that follows from the entropy (6.15). In actual fact, the approximation 
n - N » 1 does little harm - probably because the form of Stirling’s approxi- 
mation we use, ln(n-A)! = (n-A f )ln(n-A)-(n-A'), is observed not only 
when [n- N ) » 1 but also in the limit as n- N — » 0. 

Note that the number of cells n given by (6.13) is itself an extensive function 
of the fluid variables E, V, and N. And since 



(6.17) 


is the ratio of two extensive variables, the occupancy N/n is an intensive var- 
iable. When (6.17) is used to replace N/n in the normalized entropy S/Nk 
(6.15), we produce an extensive entropy function S(E,V,N). This function 
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describes all there is to know about the thermodynamic behavior of an ideal 
Fermi gas. However, we choose to keep the occupancy Njn as an auxiliary 
parameter because doing so simplifies the algebra. While the entropy (6.15) 
and the occupancy (6.17) solve the formal problem of deriving the entropy of 
the ideal Fermi gas in terms of its extensive variables, the particular properties 
of this gas remain to be investigated. 


Low occupancy regime 

When the average occupancy N/n is low, that is, when 



(6.18) 


the entropy (6.15) becomes 



(6.19) 


where in the first step of this equation sequence we have used the occupancy 
(6.17). This result (6.19) is the Sackur-Tetrode entropy (4.2) of an ideal gas. 
Occupancies for which iV/n«l define the semi-classical approximation of 
an ideal Fermi gas. 


Equations of state 


We repeat: the entropy (6.15) and the occupancy (6.17) completely describe 
the thermodynamics of an ideal Fermi gas. In particular, its equations of state 
follow from the entropy (6.15), the occupancy (6.17), l/T = (dS/dE) yN , and 
P/T = (dS/dV) E N . From these we find that 



( 6 . 20 ) 


and 
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PV 

NkT 



( 6 . 21 ) 


An immediate consequence of (6.20) and (6.21) is that 



( 6 . 22 ) 


Note that, given the occupancy (6.17), Planck’s constant h survives in the equa- 
tions of state (6.20) and (6.21) and that assuming low occupancy [N/n <§c lj 
reduces them to the familiar E - 3NkT/2 and PV = NkT. 


Low temperature regime 


How does the ideal Fermi gas behave when the occupancy N/n is relatively 
high, especially when the occupancy N/n approaches its maximum value of 1? 
We are particularly concerned to know if the relatively high occupancy regime 
is equivalent to the low temperature regime and, if so, whether the ideal Fermi 
gas observes the third law. 

While we cannot solve (6.15), (6.17), and (6.20) analytically for the func- 
tion S(T), we can use these equations to parameterize S and T in terms of the 
occupancy N/n for a given particle number N and volume V. Then we can use 
these parametric equations to plot S(t). 

In order to make our plot as meaningful as possible, we search for a nor- 
malization of T that casts Eqs. (6.17) and (6.20) into simplest form. In doing 
so we use E 0 to denote the energy realized when every one of the single par- 
ticle cells is filled with one fermion, that is when N = n, and the volume 
remains at V. According to (6.13) or (6.17) E u is the lowest possible inter- 
nal energy: the ground state energy. Thus, according to (6.13), we have 

1 = (V/ N)(E q / N) 312 (4nem/3h 2 J 12 or, equivalently. 



(6.23) 


Then (6.17) may be rewritten as 



(6.24) 


Using (6.24) to eliminate energy E from the energy equation of state (6.20) 
produces 
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Figure 6.3 Normalized entropy S/Nk (solid line) and occupancy N/n (dashed 
line) versus normalized temperature NkT/E 0 when N and V are constant for the 
ideal Fermi gas as described by (6.15), (6.17), and (6.25). 


NkT {N/nf 3 

E a ~~ UJln(l-(V/n)' 


(6.25) 


This result and expression (6.15) for the normalized entropy S/Nk allow us 
to generate values of S/Nk and N/n as a function of NkT/E a and to plot the 
result. As we see in Figure 6.3, because S/Nk — > 0 as T — » 0 for constant val- 
ues of V and N, the ideal Fermi gas observes the third law. 


Heat capacity 

The heat capacity at constant volume C v of a fluid that obeys the fundamental 
constraint dE=T dS -PdV, is related to the entropy S by 


C v 



(6.26) 


And since the entropy of an ideal Fermi gas is a function of its temperature 
through (6.15), and (6.25), we find, after some algebra, that 


C L _ 3(n/N)\n{l-N/n) 

Nk ~ [l + 3 (JV/n)/{( 1 - N/n) In (l - N/n)}] 


(6.27) 


Furthermore, given (6.22) and (6.24), the normalized pressure PV/E 0 
is given by 
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0.5 


1.0 1.5 2.0 2.5 3.0 

NkT/Eg 


Figure 6.4 The normalized energy E/E a (upper solid), normalized pressure 
PV )E 0 (dashed), and normalized heat capacity C v jNk (lower solid) of an ideal 
Fermi gas from, respectively, (6.24), (6.28), and (6.27), given the relation (6.25) 
between normalized temperature NkT/E 0 and occupancy N/n ■ The thin lines 
represent classical limits. The volume V and particle number IV are constant. 



(6.28) 


Figure 6.4 summarizes the behavior of the normalized energy E/E 0 (6.24), the 
normalized pressure PV/E 0 (6.28), and the normalized heat capacity C v /Nk 
(6.27) of an ideal Fermi gas given the relation (6.25) that parameterizes the 
normalized temperature NkT /E 0 in terms of the occupancy N/n. 


Ground state 


As the normalized temperature NkT/E u passes from values high compared 
to 1 to values low compared to 1, the ideal Fermi gas passes from its clas- 
sical regime to its quantum regime. Because, given (6.23), the normalized 
temperature is 



(6.29) 


low temperature, high particle density, and small particle mass all contribute to 
placing the ideal Fermi gas in the quantum regime. 
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In the extreme quantum limit NkT/E 0 <3C 1 the ground state pressure P 0 of 
an ideal Fermi gas is 



(6.30) 


This pressure P 0 is often called the degeneracy pressure. In this context the 
adjective degeneracy simply means “relatively high occupancy” or “in the 
quantum limit.” The phrase ground state pressure is more descriptive. A non- 
vanishing ground state energy E a , a non- vanishing ground state pressure P 0 , 
and the vanishing of the heat capacity in the T — > 0 limit are outstanding quan- 
tum features of the ideal Fermi gas. 


6.4 Average energy approximation 

The reader who consults texts listed in Appendix VI will find that the usual way 
of describing ideal Fermi gas is via the occupation number method accord- 
ing to which the entropy is maximized over occupation number macrostates 
while constrained to conserve the system energy E and particle number N. 
This approach has become standard and produces an expression for the proba- 
bility pj j^= e J that a particle of ideal gas will occupy a single-particle 

microstate whose energy is Ej. Consequently, procedures for determining the 
single-particle partition function Z l and the quantized energies Ej are required. 
Then the entropy of the gas is determined from the Gibbs entropy formula 
S = -kN y, p. In pj . 

In contrast, we have, in Section 6.3, employed and will, in Sections 7.4 
and 7.5, employ the average energy approximation according to which all the 
particles of an ideal gas have the same energy: the average energy of a particle 
E/N . Then the system entropy S[= HnQ] is determined through its multi- 
plicity fl by the number of ideal gas particles N, the number of single-particle 
microstates n those particles may occupy, and the kind of indistinguishable par- 
ticles, fermions or bosons, that compose the gas. The number of single-particle 
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microstates n proceeds from quantizing phase space into Plank’s constant- 
sized chunks and depends, according to (6.13), on the system volume V and 
average energy per particle E/N.ln this way an entropy function S{E,V, N) is 
produced that encapsulates all that can be known of the thermodynamics of a 
quantum ideal gas. 

These two methods, one based on maximizing the entropy over occupation 
number macrostates and the other based on the average energy approximation, 
produce many of the same results. Both produce the Sackur-Tetrode entropy 
in the high temperature, low density, classical limit and both produce important 
quantum features: third law compliant entropy and heat capacity and the non- 
vanishing ground state energy and pressure of an ideal Fermi gas. But there are 
differences. In particular, the ground state energy (6.23) and pressure (6.30) of 
an ideal Fermi gas predicted by the average energy approximation are a factor 
of 1.2 higher than those produced by standard methods. Quantities determined 
by the derivatives of the entropy, like the heat capacity C v , differ even in their 
functionality. 

There is no doubt that the standard approach to the ideal quantum gases 
incorporates more physics and is more accurate than the approach of the average 
energy approximation. However, we have employed the average energy approx- 
imation because it generates important results with a minimum of mathematics 
and directly exploits the properties of the entropy function S ( E,V,N ). 

Example 6.1 conduction electrons, nuclei, and white dwarfs 

The conduction electrons within a typical metal such as copper form an ideal 
Fermi gas that is essentially ground state at room temperature. Since each cop- 
per atom contributes approximately one electron to its gas of conducting elec- 
trons, the density of copper atoms, N/V ~ 8.48 x 10 28 atoms/m 3 , is the density 
of its conduction electrons. Given T = 300 K for room temperature, the nor- 
malized temperature (6.29) becomes a number, 1.15 x 10~ 12 , that is well below 
1 . Thus the conduction electrons in a metal contribute per electron much less to 
the heat capacity than that classically expected, 3k/2[= 3R/2N a \. 

The ground state pressure P 0 can be quite large for the conduction electrons 
in metals - 46.2 billion N/m 2 for copper at standard density - but is effectively 
neutralized by the electrostatic attraction between the metallic ions and the 
conduction electrons of the ideal Fermi gas. 

The nucleons (protons and neutrons) within a nucleus also form an ideal 
Fermi gas. The ground state pressure within a nucleus opposes the strong- 
force attraction between nucleons and, in this way, provides for the stability 
of the nucleus and determines its size. A similar balancing occurs between the 
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outward pressure of the electrons of white dwarf stars, stars that at the end of 
their nuclear fuel cycle have collapsed into objects a million times more dense 
than the Sun, and the inward force of gravity. 


Problems 

6.1 Extensivity 

Show that the entropy of an ideal Fermi gas is an extensive function of its 
fluid variables, that is, show that the function S [E, V, N ) implied by (6. 1 5) and 
(6.17) has the property XS(E,V,N) = S(XE, AV, A/V). 

6.2 De Broglie wavelength 

Another way to indicate when an ideal gas enters the quantum regime is to 
indicate when the de Broglie wavelength A [ = h/p\ of particles with average 
energy E/N becomes larger than the average spacing (V/N)'^ between parti- 
cle centers, (a) Derive an inequality that expresses this condition in terms of 
the average particle energy E/N, the particle density N/V, and universal con- 
stants. (b) What values of the occupancy N/n does this condition imply? 

6.3 Heat capacity 

(a) Derive the heat capacity of an ideal Fermi gas (6.27), that is, derive 
C v _ 3(n/N)ln(l-N/n) 

Nk [l + 3(A/n)/{(l-A/n)ln(l-A/n)}] 

from C v = T (dS/dT ) i/ . (b) Show that this heat capacity recovers the result 
expected for a classical, ideal, monatomic gas when N/n <sc 1 . 

6.4 Laser fusion 

One method of creating the temperatures and densities necessary for nuclear 
fusion is to shine laser beams on a pellet of the isotopes of hydrogen: deu- 
terium and tritium. The result sought is a temperature of at least 1 keV and 
an electron density of about N/V = 10 33 /m 3 in the imploded core of the fuel 
pellet. Determine the size of the normalized temperature NkT/E 0 that cor- 
responds to these parameters. Is the imploded core of the fuel pellet in the 
quantum regime? 


Problems 
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6.5 White dwarfs 

When a star consumes all its nuclear fuel it no longer generates enough energy 
to maintain its size and the star will collapse into a compact object, either a 
white dwarf, a neutron star, or a black hole, depending upon its mass. The 
ground state or degeneracy pressure of its electron gas is the mechanism that 
keeps white dwarf stars with masses up to 1.4 times the mass of the sun from 
further gravitational collapse. Consider that a spherical, homogeneous white 
dwarf of mass M and radius R will have a negative gravitational potential 
energy of U = -3GM 2 /5R, and that the inward gravitational pressure exerted 
on the surface of the white dwarf is P g = -(dt/ g /dv). A white dwarf is hot 
enough to be a fully ionized plasma and dense enough to be in the fully quan- 
tum regime. Therefore, its electron pressure is P 0 = (/V/V) 5 ’ 3 (h 2 llnern}, and 
the condition P ' + P Q = 0 establishes the equilibrium of a white dwarf. Assume 
that a white dwarf is composed of equal numbers of electrons with mass m and 
protons with mass m p . Determine how the radius R of a white dwarf depends 
on its mass M. (Hint: You will need to use the relationship between the volume 
V and the radius R of a sphere.) 
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7.1 Photons 

In 1900 Max Planck formulated an expression for the spectral energy density 
of blackbody radiation, 

, \ 8 nv 2 hv 

ai) 

that successfully reproduced the energy density of blackbody radiation con- 
tained within the differential frequency interval v to v + dv. But the physi- 
cal principles behind this formula remained obscure until illuminated during 
the next 25 years by Planck himself, by Einstein, and by Bose. Bose, in 
particular, composed all the principles behind Planck’s formula (absolute 
entropy, determinate phase space cells, quantized energy, and particle-like 
light quanta or photons) into a coherent and fully quantum theory of black- 
body radiation. 


Photons 

The idea of a light quanta or photon, introduced by Einstein in 1905, was 
at first resisted. After all, the wave theory of light had been a tremendous 
success. But the purpose of the photon concept was to make sense of a 
few phenomena that wave theory failed to explain. Among these are the 
photoelectric effect in which sufficiently high frequency light shining on a 
metallic surface ejects electrons from that surface and the Compton effect in 
which X-rays collide with free or essentially free electrons. In these effects, 
illustrated in Figure 7.1, a photon transfers some energy and momentum to 
an electron. 
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(a) (b) 

Figure 7.1 (a) The photoelectric effect: an ultraviolet photon strikes a metallic 
surface and ejects an electron, (b) The Compton effect: an X-ray photon scatters 
from a free electron. 


Complementarity 

That electromagnetic radiation sometimes behaves as if composed of waves 
and sometimes behaves as if composed of particle-like photons means that 
these very different models complement each other. Alternatively, electromag- 
netic radiation has a dual nature that is expressed quantitatively in the relation 
between the frequency v of the wave and the energy e and momentum p of the 
associated photon. 


p = kv/c. (7.3) 

Interestingly, Einstein remained skeptical of his own invention, the photon, and 
of all quantum concepts. Einstein believed that the photon is a mere heuristic, 
that is, a suggestive yet provisional device, that some day will be replaced with 
a more fundamental theory. 


7.2 Blackbody radiation 

Maxwell’s equations are linear in the electric and magnetic fields. For this 
reason electromagnetic waves, and therefore photons, do not interact with one 
another but achieve equilibrium only by being absorbed and emitted, say, by the 
walls that form the left chamber in Figure 7.2. These walls are at temperature 
T. Also in Figure 7.2 a filter allows radiation within the frequency interval from 
v to v + dv to pass from the left chamber with absorbing and emitting walls to 
the right chamber with perfectly reflecting walls. In this way the right chamber 
isolates a monochromatic or single-color part of the blackbody radiation. 

We refer to the system of monochromatic radiation composed of photons 
associated with frequency v as the subsystem throughout this section. The 
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Figure 7.2 Left chamber: blackbody radiation in equilibrium with its material 
walls at temperature T. Right chamber: monochromatic blackbody radiation 
surrounded by perfectly reflecting walls. Center: a filter that allows only single- 
frequency radiation to pass from left to right. 


photons of this subsystem have energy hv and momentum hvic and are iso- 
tropically directed, that is, directed uniformly in all directions. We adopt the 
following notation: £7 for the subsystem energy, S v for its entropy, P l for its 
pressure, Q v for its multiplicity, N p j^= E v /hv J for the number of subsystem 
photons, and N v r for the number of states that a single photon with frequency v 
may occupy. These quantities are related by S v = k\n Ll\ \/T = (dS v /E v 'j , and 
P v /T = (dS v /dv) 

Note that all quantities superscripted with v are differentials associated with 
the subsystem frequency interval v to v + dv. For instance, the energy of the 
subsystem is 

E v = V p(v)dv (7.4) 

where p(v) is the spectral energy density. The number of possible photon states 
N' c for photons with momentum p = hv/c is given by 


n; 


v 

¥ 


(momentum space volume) (number of photon polarity states) 


^[Anp 2 dp){2) 

V%np 2 dp 

h 3 

VZjrfdv 


(7.5) 


since the accessible momentum space is a shell in momentum space of radius p 
and thickness dp and each photon has two possible polarizations. 

The photon multiplicity Q v is the number of distinct arrangements of N v p 
identical photons among these N v c states. Recall the structurally identical 
problem, discussed in Section 4.4 and illustrated in Figure 4.3, of finding the 
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number of distinct arrangements of identical phonons among oscillator degrees 
of freedom. Thus, the subsystem multiplicity is 


a v = 


[n v p +K~ i)i 


(7.6) 


and therefore the absolute entropy of the subsystem is 

S" = k [(iV" + N v c ) In (AT + N v c ) - N v p lnN v p - N v c lnN v c J , (7.7) 


where we have assumed N p » 1 and » 1 . 

In developing the subsystem energy equation of state from l/T = \dS v /dE v j 
it is most convenient to start with (7.7), to keep the numbers of subsystem 
photons N v p [= E v /hv J and subsystem cells N v c j^= V8 tz\t d v/ c i \ , as parameters, 
and to proceed as follows: 


kT 


'dN' p 

dE' 


[ln(A^; + Af;) + l- 




V%xhVdv\ 

c 3 E v J’ 




(7.8) 


Solving (7.8) for E v /Vdv [= p(v)] produces Planck’s expression, 


P(v) = 


8 Jtv 


hv 


e^-l) 

for the spectral energy density of blackbody radiation. 


Example 7.1 Stefan-Boltzmann law and the radiation constant 

Problem: Derive the Stefan-Boltzmann law E/V = aT A from the spectral 
energy density of blackbody radiation and express the radiation constant a, 
first identified in Section 1.9, in terms of fundamental constants. 

Solution: Since the subsystem energy £ v is a differential associated with 
monochromatic radiation in the interval v to v + dv and the energy of a system 
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is additive over its parts, the energy E of the complete spectrum of black- 
body radiation is the integral of E“ over all frequencies from v = 0 to v = ». In 
particular. 


E = JV 

= vj~p{v)dv. 


Given the spectral energy density (7.9) we find that 
„ V8jrh r“ v’dv 


_ VXnh r~ V dv 

c 3 e hv ' kT - 1 
4 x 3 dx 


8 

15 c 3 h 3 


which is the Stefan-Boltzmann law with the radiation constant 

_ 87 r 5 k 4 
U ~~ 15 c 3 h 3 

expressed in terms of fundamental constants. 


7.3 Ideal Bose gas 


Because we approach the ideal Bose gas in much the same way as we approached 
the ideal Fermi gas in Section 6.3, we should recall the first few paragraphs of 
that section. Here we also identify the number of single-particle microstates 
or cells available to a particle of ideal gas with energy E/N, whether distin- 
guishable, fermion, or boson by 




(7.10) 


where E, V, and N are the extensive fluid variables that describe the sys- 
tem, h is Planck’s constant, and e is the base of the natural logarithm. And 
we again make use of the average occupation number per cell, that is, the 
occupancy 
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(7.11) 


Because any number of bosons may occupy the same single particle cell, 

0< — <N (7.12) 

n 

defines the domain of the occupancy of an ideal Bose gas. And we again adopt 
the average energy approximation. 


The multiplicity of an ideal Bose gas 

The number of distinct ways of placing N identical bosons each with energy 
E/N in n distinct cells is the multiplicity 

n(E - v - w)=i viSf' (7 - 13) 

The counting that produces (7.13) is structurally identical to counting the 
number of distinct ways of ordering a collection of N identical balls and n - 1 
identical white dividers. When placed in a row, the n- \ dividers separate the 
N balls into n ordered (and so distinct) groups or cells. See, for comparison, 
Figure 4.3. Also, recall that we used the combinatorial formula (7.13) when 
counting phonon microstates in Section 4.4 and when counting photon micro- 
states in Section 7.2 - both examples of counting bosons. 

Paul Ehrenfest (1880-1933) questioned Einstein about replacing the mul- 
tiplicity n N of an ideal classical gas of distinguishable particles with the mul- 
tiplicity (A + n-l)!/[(n-l)!A!] of an ideal gas of indistinguishable bosons. 
For Q = n N implies that the particles of an ideal classical gas occupy their 
position in single-particle phase space independently of one another. After all, 
if Q = n N then the system entropy S [= Nk In nj is simply the sum of the entro- 
pies of the individual particles. However, if Q = (iV + n-l)!/[(n-l)!lV!], 
no such decomposition of the multiplicity and no such interpretation of the 
entropy is possible. Apparently, the particles of an ideal Bose gas do not 
occupy their positions in phase space independently of one another. While 
Einstein did not object to this characterization he found it inexplicable. That 
the particles of an ideal Bose gas, and likewise the particles of an ideal Fermi 
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gas, are correlated with one another even in the absence of interparticle forces 
is a purely quantum effect. 


The entropy 

Given (7.13) the entropy of an /V-particle ideal Bose gas is given by 

S(£ > V,iy) = tln ^”^ ) i 1 j. (7.14) 


We assume that N » 1 and n » 1 in order to deploy Stirling’s approximation 
and so from (7.14) generate 


S(E,V,N) _ 


1 + — ln(N + «)-lnJV- 


= tt In 1 + - +ln 1 + - . 


Note that the entropy (7.15) of an ideal Bose gas is, given (7.11), an extensive 
function of its extensive variables E, V, and N. While Eqs. (7.11), (7.14), and 
(7.15) describe all there is to know of the thermodynamic behavior of an ideal 
Bose gas, the particular properties of this gas remain to be explored. 


Low occupancy regime 

When the occupancy N/n is low, that is, when 
N 

— «1, (7.16) 


the leading-order terms in an expansion of the entropy (7.15) are 



(7.17) 


where in the first step we have used (7.11). This result (7.17) is the Sackur- 
Tetrode entropy (4.2) of an ideal gas. 
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Equations of state 

The entropy S as given by (7.15) and the occupancy N/n as defined by (7.11) 
completely describe the thermodynamics of the ideal Bose gas as long as 
N » 1 and n»l. In particular, its equations of state follow from (7.11), 
(7.15), l/T = (dS/dE) v N , and P/T m (dS/dV) b , jy . From these we find that 


and 



(7.18) 


(7.19) 


An immediate consequence of the equations of state (7.18) and (7.19) is that 

£ = | PV. (7.20) 

Note that Planck’s constant h survives in these equations of state through its 
appearance in the occupancy (7.11) and that the low occupancy limit reduces 
the equations of state (7.18) and (7.19) to the familiar ideal gas equations of 
state PV = NkT and E = 3NkT/2. 


Low temperature regime 

How does the ideal Bose gas behave in the low temperature regime? Expression 
(7.11) insures that the high occupancy regime is equivalent to the low energy 
regime while (7.18) insures that the low energy regime is equivalent to the low 
temperature regime. Therefore, low temperature means high occupancy for the 
ideal Bose gas just as it did for the ideal Fermi gas. 

We are particularly concerned to know if the ideal Bose gas described by 
the equations of state (7.18) and (7.19) observes the third law. Since we cannot 
solve (7.11), (7.15), and (7.18) analytically for the function S [t), we again use 
the occupancy N/n as a parameter that numerically links normalized values of 
the entropy S and temperature T for constant values of N and V. We plot the 
resulting function S(t) in Figure 7.3. However, the parameters with which 
we normalize the temperature of the ideal Bose gas cannot be those used to 
normalize the temperature of the ideal Fermi gas. After all, the two gases have 
very different ground states. 
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A particular ground state of the ideal Bose gas is one in which all N 
bosons occupy the same single-particle state, and so n = 1, and yet its vol- 
ume remains at V. We find that the energy of this special ground state, 
denoted E w , is a convenient normalizing energy. In this case (7.10) reduces to 

1 = v i E oo /uf 2 Unem/3h 2 f 2 . Therefore, 


E oo 


N 

V 2/3 


— 1 
4k em J 


(7.21) 


is the energy of this ground state in terms of which (7.10) may be expressed 


E 

A 2/3 £ 00 



(7.22) 


Using (7.22) to eliminate the energy E from the energy equation of state (7.18) 
produces a parametric equation, 


kTN 1 ' 3 ( N/n) W 

E w - UJln(l + tf/n)’ 


(7.23) 


for the normalized temperature kTN' : ' i / E m . This result (7.23) and the normal- 
ized entropy S/Nk given by (7. 15) allow us to numerically generate coordinate 
values of S/Nk and kTN 1/3 /E m and plot the result in Figure 7.3. The occupan- 
cies that generate Figure 7.3 range from N/n = 1 (upper right) to N/n = 100 
(lower left center). 

Note the surprising shape of S (T). The .S' (7’) curve bends around and forms 
a nose near N/n = 16 at which point the slope (dS/dT) y diverges to infinity. 
These features signal instability. But before we investigate this instability, we 
consider in more detail the special n = 1 ground state in which all the system 
bosons occupy the same single-particle microstate. 


Ground state 

It is important to realize that the n = 1 ground state is not described by the 
entropy (7.15) nor by the equations of state, (7.18) and (7.19), that follow from 
it, nor by any equations that follow from these. After all, these equations result 
from applying Stirling’s approximation to S = A:lnQ and so assume n»l as 
well as N » 1. In describing the n = 1 ground state we depend solely on (7.1 1)- 
(7.13) and the combinatoric expression for entropy (7.14). Therefore, the n = 1 
ground state multiplicity is Q = 1, and consequently the ground state entropy 
5 = 0. Furthermore, this ground state is distinct from the macrostates illustrated 
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ktn V 3 /e 00 


Figure 7.3 The normalized entropy S/Nk versus the normalized temperature 
kTN l/> / E lyl when N and V are constants for occupancies ranging from N/n = 1 
(upper right) to N/n = 100 (lower left-center) as determined from (7.15) 
and (7.23). 


in Figure 7.3. In principle, the n = 1 ground state with energy denoted E 0 is dis- 
tinct from the normalizing ground state energy E m because the n = 1 ground 
state volume V 0 can be different from the volume V of the n » 1 macrostates. 


Thermodynamic instability 

Because the fundamental equation dE = TdS-PdV implies that the heat 
capacity at constant volume C v f = (dE/dT ) y J is given by T (dS/dT) y , a negative 
slope (dS/dT) y indicates a negative heat capacity. And a system with negative 
heat capacity cannot be in equilibrium with a constant-temperature environment. 
After all, the temperature of a system with negative heat capacity would decrease 
when the system was heated and increase when cooled. Either behavior would 
drive the system further from equilibrium with a constant-temperature environ- 
ment. Therefore, according to the calculation displayed in Figure 7.3, the ideal 
Bose gas is unstable for occupancies greater than about N/n ~ 16. 

Actually, the ideal Bose gas is thermodynamically unstable for much 
smaller occupancies. To see this, consider that an isolated system is 
thermodynamically unstable whenever it can transition to another macrostate 
with higher entropy. 
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In actual fact experimental realizations of low temperature ideal Bose gases 
are never isolated and a path always exists for the system to lose energy and 
occupy its ground state. While experiments vary, typically the gas is in contact 
with an environment at constant temperature T and constant pressure P as T is 
slowly decreased. In this case, the ideal Bose gas will begin to occupy its n = 1 
ground state when by doing so it can decrease its Gibbs free energy. In particu- 
lar, the Gibbs free energy G [= E - TS + PV \ of the n > 1 state begins to exceed 
the Gibbs free energy of the n = 1 ground state whenever 


G n>t = G n=1 , 

E-TS + PV = E 0 -TS 0 +PV 0 , 
-E-TS — O, 



where in the second step of this equation sequence we use (7.20) on the left- 
hand side and S 0 = 0 on the right hand side and recognize that the ground state 
energy and volume, E 0 and V 0 , are ignorably small compared with E and V. 

The critical occupancy that solves (7.24) is 

— = 0.392. (7.25) 

Whenever G n>1 > G n=1 the occupancy N/n>N/n c and the ideal Bose gas 
becomes thermodynamically unstable. Alternatively, whenever N/n<N/n c 
the ideal Bose gas is thermodynamically stable. This critical occupancy N/n c 
separating stability from instability corresponds, according to (7.23), to a nor- 
malized critical temperature k TN l!l / E w = 1.48 or, equivalently, to 


kT 



(7.26) 


a result that is a factor of 1.5 greater than the critical temperature predicted 
by models that incorporate more physics. The critical point, for which 
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kT c N l/3 /E m = 1.48, SjNk =2.11, and N/n c = 0.392, is on the curve shown in 
Figure 7.3 but far beyond the figure’s upper right hand corner. 


7.4 Bose-Ein s tein condensate 

If we cool an ideal Bose gas to a temperature just below its critical temperature 
T and, consequently, to an occupancy just above its critical occupancy N/n 
[= 0.392], the gas will break into two parts with each part being expelled to a 
nearby relative minimum in the Gibbs free energy, that is, to a stable region of 
thermodynamic phase space. When the temperature of the gas remains close 
to its critical temperature, the largest share of the particles will remain at the 
critical occupancy N/n c and the remaining, smaller share will condense into 
the n = 1,5 = 0 ground state. This transition is called a phase transition and 
the material in each share is a called a different phase of the gas. Thus, when 
T < T the system becomes a composite, two-phase system. The phase remain- 
ing at the onset of instability is a normal, if low temperature, ideal Bose gas 
while the phase in the ground state is one whose macrostate consists of one 
single-particle microstate. 

Albert Einstein first predicted the existence of the n = 1 phase or conden- 
sate early in 1925. Einstein, who was inspired by Bose’s 1924 analysis of 
blackbody radiation, had applied Bose’s counting method, that is, Bose’s 
statistics, to an ideal gas composed of identical material particles. Then, on 
the basis of an extended analysis, Einstein predicted the existence of this 
condensed, n = 1, single-particle microstate phase we now called the Bose- 
Einstein condensate. 


Two-phase regime 

As the system temperature T drops below its critical temperature T the number 
of particles occupying the uncondensed phase N un decreases and the number 
occupying the n = 1 ground state phase N 0 increases in such a way that the total 
number of particles 

N = N m + N 0 (7.27) 

is conserved. Alternatively, the fraction of particles in the uncondensed phase 
N un /N decreases and the fraction of particles in the ground state phase N 0 /N 
increases as the temperature T drops further below T c . The system energy and 
entropy are, however, not conserved. Both decrease as the system temperature 
T decreases to below T . 
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Because the gas particles always seek the closest stable macrostate consistent 
with its current temperature T, that is, seek the state with the smallest Gibbs free 
energy, the N un uncondensed particles remain at critical occupancy so that 

= -— [= 0.392] (7.28) 

as both N un and n un decrease. Thus, condition (7.28) expresses the stability of 
the particles in the uncondensed phase. 

This stability relation (7.28) has important consequences. Using the defini- 
tion of the occupancy (7.11) to expand both sides of (7.28) we find 

iy m n 



' 3 h 2 y 

'em J 



_ N 512 
E ^ ~ Ef 


(7.29) 


where E un is the energy of those particles that remain in the uncondensed, criti- 
cal state. Similarly we compare the energy equation of state (7.18) when T = T c 
and, consequently, all the particles are at the critical point, 


E c 

NkT c 



(7.30) 


with the energy equation of state (7.18) when T < T c and only part of the uncon- 
densed particles are at the critical point, 


E uu 

NJiT 



(7.31) 


Given the stability condition (7.28) the right-hand sides of Eqs. (7.30) and 

(7.31) are equal. Therefore, 


E m _ E c 
N m T ~ NT 


(7.32) 


when T < T c . Eliminating the ratio E un /E c from (7.29) and (7.32) produces 


(7.33) 
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which, given particle conservation (7.27), is equivalent to 



(7.34) 


an expression that describes the way in which the ground state is populated as 
the temperature T drops below the critical temperature T and approaches abso- 
lute zero. According to (7.34), when T = T c all the particles are in the criti- 
cal, uncondensed phase and when T = 0 all the particles are in the condensed, 
ground state phase. 

The entropy of the composite, two-phase system is the sum of the entropies 
of its two phases. Since the entropy of the ground state phase vanishes, we have 
for the entropy of the composite system 


= 2.11 N un k, 


where in the first and second steps we have used the stability condition (7.28). 
Given (7.35) and (7.33), the normalized entropy of the two-phase system is 



(7.36) 


Figure 7.4 replots Figure 7.3 with the unstable part of the S - T relation sub- 
tracted and the two-phase part of the S - T relation (7.36) added. The solid curve 
shows the entropy. Since S — > 0 as T — > 0, this composite system observes the 
third law. Also shown (dashed curve) is the normalized heat capacity C v /Nk. 
(See Problem 7.7.) Because the entropy is continuous at the critical tempera- 
ture while its first derivative with respect to temperature ( dS/dT ) y [= C v /T ] is 
discontinuous, this transition, at least in the average energy approximation, is 
a first-order phase transition. 

Albert Einstein remarked in a letter to Paul Ehrenfest, “The theory [of 
the ideal Bose gas] is pretty, but is there some truth to it?” The world had 
to wait some years for experimental verification of Einstein’s prediction of a 
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KTN^/E 00 

Figure 7.4 Normalized entropy S/Nk (solid curve) versus the normalized 
temperature kTN l ‘/ for densities ranging from N/n = 0.3 (upper right) to 
the n = 1 ground state (lower left). Determined from (7.15) and (7.23) when 
T > T c and from (7.23) and (7.36) when T <T c . Also shown is the normalized 
heat capacity C v = T(dS/dT) (dashed curve) versus normalized temperature as 
taken from Problem 7.7. 


low temperature condensed phase of an ideal Bose gas. In 1938 Fritz London 
interpreted the phase transition between helium I and helium II as a transition 
to a Bose-Einstein condensate. The observed critical temperature was lower 
than predicted, but helium atoms attract each other at the liquid densities at 
which the condensation was observed and these attractive interatomic forces 
would explain this discrepancy. It was not until 1995 that Eric Cornell and 
Carl Wieman of the University of Colorado cooled a gas of weakly interact- 
ing bosons (rubidium atoms) below the critical temperature and produced a 
Bose-Einstein condensate. For this work they, along with Wolfgang Ketterle 
for shortly thereafter creating a Bose-Einstein condensate with sodium atoms 
at the Massachusetts Institute of Technology, received the 2001 Nobel Prize 
in Physics. 


7.5 Modeling the ideal gas 

We reflect on our derivations of the ideal Fermi and ideal Bose gases in 
Sections 6.3 and 7.3. The quantum properties incorporated into these mod- 
els include: phase space cells determined by Planck’s constant h, the absolute 
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entropy S [= &lnQ], the indistinguishability of identical particles, and the par- 
ticular ways we count boson and fermion microstates. Thus, these models do 
not depend upon quantizing dynamical properties like the particle energy or 
momentum nor on assuming a particular shaped volume that makes such quan- 
tizing convenient. 


Average energy approximation 

We also used the average energy approximation in describing the ideal Fermi 
and ideal Bose gases. The average energy approximation shifts the ground 
state energy and pressure of the ideal Fermi gas by a factor of approximately 
1 .2 and the critical temperature of the ideal Bose gas by a factor of approx- 
imately 1.5 above values predicted by more complete, standard derivations. 
Furthermore, the average energy approximation distorts the derivatives of the 
entropy function in the low temperature, quantum regime. Nevertheless the 
average energy approximation allows us to quickly reproduce the distinguish- 
ing features of the ideal quantum gases: entropies and heat capacities that van- 
ish in the T — > 0 limit, non- vanishing ground state energy and pressure of the 
ideal Fermi gas, and a critical temperature below which the ideal Bose gas 
becomes two phase. 


Quantum and classical descriptions 

In summarizing the differences between the ideal classical and ideal quantum 
gases we place those properties associated with a quantum description in the 
left column and those properties associated with a classical description in the 
right column. Note that we include extensivity in both columns and that consis- 
tency with the Sackur-Tetrode entropy in the semi-classical or low occupancy 
limit is the only property that applies exclusively to ideal gases. 


Quantum description 


Classical description 


Entropy is extensive 
Identical particles are indistinguishable 
Entropy is third law compliant 
Absolute entropy S [= k In Q] 
Sackur-Tetrode entropy when occupancy 
Mn«l 

H becomes Planck’s constant h 
h survives in equations of state 


Entropy is extensive 
Identical particles are distinguishable 
Entropy is not third law compliant 
Relative entropy S [= c + k In Q] 
Consistent with Sackur-Tetrode 
entropy 
H is arbitrary 

H vanishes from equations of state 
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Problems 

7.1 Radiation pressure 

(a) Derive an expression for the spectral pressure of blackbody radiation P v in 
terms of the radiation frequency v, the system temperature T, and universal con- 
stants. (Hint: Start with the spectral entropy S v (7.7), use P" /t = {dS'/dV^ , 
proceed as in deriving (7.9), and eliminate £ v with E v /v dv = p(v).) (b) 
Integrate P v from v = 0 to v = °° to get the total pressure P in terms of T and 
universal constants. (You will need to complete the integral with software or 
look it up in Appendix H: Formulary.) (c) Show that the pressure P and internal 
energy E of blackbody radiation are related by P = E/3V. 

7.2 Radiation pressure at the center of the Sun 

The temperature at the center of the Sun is approximately 2 x 10 7 K. Find the 
corresponding blackbody radiation pressure. What is the ratio of this radiation 
pressure to the gas pressure (4 x 10 11 atmospheres) at the center of the Sun? 
(Hint: consult Problem 7.1 or reread Section 1.9.) 


7.3 Maximum radiation 

The energy of blackbody radiation within the frequency range v to v + dv is 
p(v)dv where p(v) is the spectral energy density given by (7.1) or (7.9). (a) 
Given that the frequency v and wavelength X of electromagnetic radiation in a 
vacuum are related by c = Xv, find the spectral energy density per differential 
wavelength e(A) defined so that e{X)dX = p(v)dv. (b) Find an expression, 
called Wien’s law, for the wavelength A max that maximizes the spectral energy 
density per differential wavelength s(X). (You will have to numerically solve 
a simple equation.) (c) Given that the Sun radiates the most energy at a wave- 
length of close to 5 x 1(T 7 m, what is the effective temperature of the surface 
of the Sun? 


7.4 Number of photons 

Find an expression for the number of blackbody radiation photons as a func- 
tion of the volume V and temperature T. (Hint: Integrate E v /hv from v = 0 to 

v = You will have to numerically evaluate the integral £ x / (e x -\)dx or 
see the Formulary in Appendix II.) 


Problems 
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7.5 Two-phase energy 


The context is Sections 7.3 and 7.4 on the Bose-Einstein gas and condensate. 
Assume that the particles in the condensed state make an ignorably small con- 
tribution to the total energy of the system. From (7.18) find an expression the 
energy E of the composite system in terms of N, T and T A in the two-phase 
region, that is, when T < T c . 


7.6 Photon condensate? 


A cavity full of electromagnetic radiation is a gas of photons, and photons are 
bosons. Why, then, do photons never condense at low temperatures? 

7.7 Heat capacity of ideal Bose gas and condensate 

(a) Show that the heat capacity of the ideal Bose gas when T > T c is given by 


Cy _ (n/N){l + N/n)[ln{l + N/n)f 
Nk N/n-(l/3)(l + N/n)]n(l + N/n)' 


(b) Show that the heat capacity C v =T (dS/dT) y of the composite ideal Bose 
gas in the two-phase region where T < T. is given by 



These expressions are used in the construction of the dashed curve in 
Figure 7.4. 
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8.1 Messages and message sources 

Information technologies are as old as the first recorded messages, but not until 
the twentieth century did engineers and scientists begin to quantify something 
they called information. Yet the word information poorly describes the concept 
the first information theorists quantified. Of course specialists have every right 
to select words in common use and give them new meaning. Isaac Newton, for 
instance, defined force and work in ways useful in his theory of dynamics. But 
a well-chosen name is one whose special, technical meaning does not clash 
with its range of common meanings. Curiously, the information of information 
theory violates this commonsense rule. 

Compare the opening phrase of Dickens’s A Tale of Two Cities: It was the 
best of times, it was the worst of times . . . with the sequence of 50 letters, 
spaces, and a comma: eon jhktsiwnsho d ri nwfnn ti losabt, tob euffr te ... taken 
from the tenth position on the first 50 pages of the same book. To me the first is 
richly associative; the second means nothing. The first has meaning and form; 
the second does not. Yet these two phrases could be said to carry the same 
information content because they have the same source. Each is a sequence of 
50 characters taken from English text. 

Information theorists and communications engineers are appropriately con- 
cerned with the ability to transmit messages of a given form rather than with 
the content of particular messages. According to Claude Shannon (1916-2001), 
whose 1948 paper “The Mathematical Theory of Communication” initiated the 
study of information theory. 

These semantic aspects of communication are irrelevant to the engineering problem. 
The significant aspect is that the actual message is one selected from a set of possible 
messages. The system must be designed to operate for each possible selection, not just 
the one which will actually be chosen since this is unknown at the time of design. 
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The set of possible messages or symbols from which an actual message is 
selected is a message source that contains or produces messages. Information 
theory is concerned with characterizing message sources rather than with char- 
acterizing particular messages. 


8.2 Hartley’s information 

Ralph V. L. Hartley (1888-1970) was the first, in 1928, to quantify the infor- 
mation content of a message source with just two numbers: n, the number of 
characters in the equal-length sequences that compose its messages, and s, 
the number of equiprobable symbols that each character may assume. Note 
that the s" distinct messages in Hartley’s message source must themselves be 
equiprobable since they are composed of equal numbers of equally probable 
symbols. While Hartley’s s symbols could be letters of the alphabet or dots and 
dashes or the natural numbers, no violence is done his logic if the symbols are 
themselves equiprobable messages each considered as a unitary whole. 

Hartley required that the measure of information H of such a message 
source have two properties. First, H must be proportional to n, the number of 
characters in each message sequence. Thus, 

H=nf(s). (8.1) 

And second, H must be a monotonically increasing function, 

H = g{s n ), (8.2) 

of the number s n of distinct, equiprobable messages the source contains. After 
all, a message source containing messages with twice as many characters 
should contain twice the information and a message source containing more 
messages should contain more information. 

According to requirements (8.1) and (8.2), 

nj{s) = g(s") (8.3) 

where fix) and g(x) are monotonically increasing functions on the domain 
x > 0 . The only differentiable, monotonically increasing functions fix) and 
g(x) that satisfy (8.3) are 

fix) = g(x) 

= c In x, (8.4) 

where c is an arbitrary positive constant. (Problem 8.1 outlines a proof of this 
statement.) Thus, Hartley defined the information content of a message source 
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containing messages with n characters each character of which may be realized 
with s equally probable symbols as 

H = c\ns". (8.5) 

Equation (8.5) neatly illustrates a general principle: The information content 
of a message source is proportional to the logarithm of the number of distinct, 
equiprobable messages it contains. 

No experiment can determine the value of the constant c. However, the 
constant c can be used to determine when H = 1, that is, c can be used to 
determine the unit of information. When c = 1, Hln = In s and H is measured 

in nats (after natural logarithm). Hartley used c = 1/ln 10 in his 1928 paper. 

In this case, 

H _ Ins 

7 ” lino (8.6) 

= log 10 S- 

Thus a message source composed of the numerals 0, 1, 2,... and 9 has s = 10 
and so is said to have one Hartley of information content per message char- 
acter. The choice c = 1/ln 2 is, for reasons that will soon be apparent, very 
convenient. Then 

H _ Ini 

~n ~ W2 (8.7) 

= log 2 i, 

and Hln is measured in bits (a contraction of binary digits). A message source 
composed of all possible messages of n characters each character of which can 
be realized with only 2 [= s] equiprobable characters contains 1 bit of infor- 
mation per character. In general the constant c determines the base b of the 
logarithm. When c = 1/ln b,H= log ft s n . 


An application 

Occasionally one hears that a particular message contains so many bits of 
information. For instance, “01 1100 contains 6 bits of information.” To be more 
correct one should say that the message 011100 appears to come from a mes- 
sage source composed of 6-character sequences of equally probable zeros and 
ones. In this case, n = 6, s = 2, and, according to (8.5), the information content 
of the message source that produced the message 011 100 is 6 [= 6 log 2 2] bits, 
4.16 [= 6 In 2] nats, or 1.81 [= 6 log 10 2] Hartleys. 


8.3 Information and entropy 
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The symbol H 

Why did Hartley adopt the symbol H for information? We simply do not know. 
That Hartley wanted to memorialize his own name seems unlikely. He may have 
thought that c In s n was structurally similar to the H of Boltzmann’s celebrated 
H theorem. But, if so, Hartley was at least partly mistaken. For Hartley’s H is, 
apart from normalizing constants, the negative of Boltzmann’s H. Thus, the 
larger the Boltzmann H of a message source, the smaller its information H. 

Example 8.1 Shuffled deck 

Problem'. A 52-card deck is completely shuffled, (a) What is the information 
content of the shuffled deck taken as a whole? (b) What is the information con- 
tent of a 5-card poker hand drawn from the shuffled deck? 

Solution : Here we appeal to the general principle that the information con- 
tent of a message source is proportional to the logarithm of the number of dis- 
tinct, equiprobable messages it contains. Consider the possible arrangements 
of the shuffled deck and of the poker hand. Since each of these arrangements 
is equally probable we need only count their number: 52! different equally 
probable arrangements of the shuffled deck and 52 x 5 1 x 50 x 49 x 48 different 
equally probable arrangements of the 5-card poker hand. Therefore, the infor- 
mation content of the shuffled deck is log 2 52! or 226 bits and the information 
content of the 5-card poker hand is log 2 (52 x 51 x 50 x 49 x 48) or 22.6 bits. 


8.3 Information and entropy 

Hartley’s information measure c In s n is comparable not to Boltzmann’s H but 
rather to the absolute entropy kin 12 of an isolated system. The two measures, 
information and entropy, are similarly constructed. Other comparisons can also 
be made including: 


a symbol 

with 

a microstate of a simple system 

a single-character message 

with 

a simple system 

a character sequence 

with 

a complex system 

a particular message 

with 

a microstate of a complex system 

a message source 

with 

an ensemble of complex systems 

the number of equiprobable 

with 

the number of equiprobable 

messages 


microstates 

the constant c 

with 

Boltzmann’s constant k 

the information of a message source 

with 

the entropy of an isolated system 
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But a deep analogy does not imply identity. And a single name, either entropy or 
information content for the two concepts, insufficiently distinguishes between 
a message source and an ensemble of physical systems. After all, physical 
entropy is a consequence of the laws of thermodynamics and obeys these laws, 
while information content is a descriptive measure with certain built-in proper- 
ties. Apparently the two concepts are substantively distinct if formally similar. 


Missing information 


A third term, missing information, more appropriately references the parallel 
structures of the entropy of physics and chemistry and the information of infor- 
mation theory. Hartley’s information measure cln.s" is actually the missing 
information needed to select one n-character message each character of which 
can be replaced with s symbols out of a message source containing all possible 
such messages. And if we divide the physical entropy S by Boltzmann’s con- 
stant k, the result S/k [= In Q] is also the missing information needed to specify 
the microstate of a physical system. 


Example 8.2 Missing information 


Problem: What is the missing information per atom of a three-dimensional 
Einstein solid composed of carbon atoms in diamond form at a temperature 
of 300 K? 

Solution: This problem is equivalent to that of finding the value of S/Nk 
implied by the entropy of the Einstein solid (4.14), 



with its energy (4.13), 



for parameters characteristic of diamond (hv o lk= 1325 K) at temperature 
T = 300 K. We must also set c(N ) equal to zero because only the missing infor- 
mation of a third law compliant physical system is comparable to the informa- 
tion content of a message source. We find that hv a IkT - 1325 / 300 = 4.42 , 
E/3Nhr a = 0.512 , and, therefore, S/Nk = 0.196 bits of missing information 
per carbon atom. 


8.4 Shannon entropy 
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8.4 Shannon entropy 

In 1948 Claude Shannon generalized Hartley’s information measure and 
searched for a name to replace the potentially misleading information. After 
considering and rejecting choice and uncertainty. Shannon settled on entropy - 
no doubt because of the deep analogy between the entropy of physics and 
chemistry and the information of communications theory. Recall that Hartley’s 
information H [= c In s n ] describes the entropy of a message source composed 
of equally probable messages or symbols. Such a message source is analogous 
to an ensemble of isolated systems with equally probable microstates. But just 
as a non-isolated physical system may occupy its states with unequal probabil- 
ities, a message source more complicated than Hartley’s may be defined that 
realizes its symbols with unequal probabilities. 

Consider a message composed of the 26 letters of the English alphabet. 
According to Hartley, the information content per character H/n of a message 
composed of these symbols is log 2 26 or 4.70 bits - assuming the 26 letters 
of the English alphabet appear with equal probability p [= 1/26 = 0.0385], 
But a brief examination of English text reveals that the letters of the alphabet 
do not appear with equal probability. In particular, the letter e appears most 
frequently, in fact, typically with frequency 0.13. The letter t appears with 
frequency 0.091, j with frequency 0.015, and z with frequency 0.0074. (See 
Table 8.1.) If we want to describe a message source that produces text with a 
frequency of symbols characteristic of English text, we cannot assume that the 
letters of the alphabet are equally probable. 

Constructing the Shannon entropy 

Shannon generalized Hartley’s information content by constructing an infor- 
mation measure that allows for symbols with unequal probabilities. In reformu- 
lating Shannon’s argument we seek the missing information or entropy H{n v 
n 2 , n) of a message source composed of all possible n-character sequences 
for which n = + n 2 + ■■■ + n s that, in turn, are composed of s symbols the 

zth one of which occurs n. times. For this purpose we consider only very long 
(n » 1) equal-length messages in which the zth symbol appears with a fre- 
quency nJn that closely matches its probability so that 

/>, = -, ( 8 . 8 ) 


where, of course. 
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v .... . < 89 > 

and ^ Pi = 1. Since each long, equal-length message in the source is equi- 


(8.9) 


probable even if the symbols that compose the messages appear in each mes- 
sage with unequal probabilities, we can apply the general principle suggested 
by Hartley’s analysis: The missing information or entropy of a message source 
is proportional to the logarithm of the number of distinct, equiprobable mes- 
sages it contains. Since n !/ (n, ! n 2 ! ■ • • n s !) is the number of distinct, equiproba- 
ble ^-character sequences that can be formed out of n } occurrences of the first 
symbol, n 2 occurrences of the second symbol and so on, we have 


H(n.,n 2 , ra ) = cln — 

L W-V. 


( 8 . 10 ) 


Secondly, we require that the missing information or entropy H{n v n 2 , . . . n) 
of an ^-character sequence be n times the entropy of a one-character sequence 
H(p v p 2 , p 3 ) governed by the same probabilities, that is, 


H{n x , n 2 , ..., n s ) = H(np v np 2 , ..., np s ) 
= nH(p i, p 2 , ..., pj, 


( 8 . 11 ) 


where the n and the p. are related by (8.8). Requirement (8.11) is a general- 
ization of Hartley’s requirement (8.1) and also is analogous to the requirement 
that the physical entropy be an extensive function of its variables. 

The two requirements (8.10) and (8.11) together imply that 




( 8 . 12 ) 


•{nlnn-X 



-C ']L ,' =1 P, ]n Pn 
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where the first line of this equation sequence is from (8.11), the second uses 
(8.10), the third assumes that n. » 1, and the last exploits (8.8). The missing 
information or entropy of a message source composed of one-character mes- 
sages realized with unequal probabilities, 

H{, Pi, P 2 , P s ) = -^YJ i .Pi Xn Pi’ (8 ' 13) 


is called the Shannon entropy. 


Reduction to expected results 

The Shannon entropy (8.13) reduces to expected results in appropriate limits. 
For instance, when all the symbols are equiprobable so that p x =p 2 = ■■■ =p s = 
1 1s, the result 



= c In .s' 


recovers Hartley’s information content for a one-character message realized 
with s equiprobable symbols. When one symbol is certain with probability 1 
and the others appear with probability 0, the Shannon entropy 

H{\, 0, -, 0) = -c{lxlnl + 0xln0 + 0xln0+ - }, (g ^ 
= 0 

where here we stipulate that 0 x In 0 = 0 in order to recover the limit of x In x 
as x — » 0. Thus, a message source that invariably produces the same symbol or 
message has zero Shannon entropy. 


Example 8.3 Two symbols 

Problem: A message source produces two symbols, the first with probability 
p and the second with probability q [= 1 - p] . What value of p maximizes the 
Shannon entropy of this message source? 

Solution : The Shannon entropy of this one-character, two-symbol source is 

H ( p , q) = ~p log 2 p - q log 2 q 

= -p\og 2 p-(\-p) log 2 (1 - p) 

when measured in bits. The value of p that maximizes H{p, 1 - p) is the 
solution of 

— log 2 p - 1 + log 2 (l - p )+ 1 = 0, 
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P 

Figure 8.1. The Shannon entropy H(p, 1 — p) versus p of a source that produces 
two symbols with probabilities p and 1 - p . Used in Example 8.3. 

that is, p = l/2. The graph of H(p, 1 - p) shown in Figure 8.1 confirms 
that p - 1/2 is a maximum. Therefore, the maximum Shannon entropy is 
H{ 1/2, 1/2) = log 2 2 or 1 bit. 

Example 8.4 Entropy of English text 

Problem : What is the Shannon entropy of a source that produces single charac- 
ters of the alphabet with frequencies observed in English text? 

Solution: The frequencies of the letters of the alphabet in English text (dis- 
played in Table 8. 1) are easily found online. We assume that these frequencies 
approximate probabilities. The longer the texts in which the frequencies are 
observed, the more accurate is this assumption. The calculation of the Shannon 
entropy per character of English text is then straightforward if tedious. Thus, 

(0.0817, 0.0149, •••) = - 0.0817 x log 2 (0.08 17) - 0.0149 x log 2 (0.0149) ■ • ■ 

= 4.47 bits, 

a value somewhat smaller than the 4.70 hits per character that would be pro- 
duced if all 26 letters were equally probable. Apparently, the letters in English 
text appear with less uncertainty, less missing information, and less Shannon 
entropy than they would if equally probable. 

8.5 Fano code 

One application of Shannon entropy is to evaluate the efficiency of coding 
one alphabet of symbols, in terms of which an original message source is 
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Table 8.1. Frequency with which the letters of the alphabet appear in English text. 


Letter 

Probability 

Letter 

Probability 

Letter 

Probability 

a 

0.082 

j 

0.001 5 

s 

0.063 

b 

0.015 

k 

0.0077 

t 

0.091 

c 

0.028 

1 

0.040 

u 

0.028 

d 

0.043 

m 

0.024 

V 

0.0098 

e 

0.13 

n 

0.067 

w 

0.024 

f 

0.022 

0 

0.075 

X 

0.001 5 

g 

0.020 

P 

0.019 

y 

0.020 

h 

0.061 

q 

0.00095 

z 

0.00074 

1 

0.070 

r 

0.060 




composed, into another alphabet of symbols, typically binary digits, conve- 
nient for electronic transmission. The more closely the Shannon entropy of the 
coded message source matches the Shannon entropy of the original message 
source, the more efficient the code. 

Suppose, for instance, a message source produces the symbols a, b, c, and 
d with the probabilities 0.5, 0.2, 0.2, and 0.1 as described in Table 8.2. The 
Shannon entropy per character of this source is 

- 0.5 x log 2 0.5 - 2 x 0.2 x log 2 0.2 - 0. 1 x log 2 0.1 = 1 .76 bits. (8.16) 

The coder’s task is to translate the letters a, b, c, and d into combinations of 
transmissible binary digits 0 and 1. Obviously, the most frequently occurring 
letter, a, should be represented by the shortest sequence of binary digits and 
so on, but how exactly? Fano code is one method. The goal of Fano code is to 
use log 2 (l/ p t ) binary digits to represent the ith symbol. Since log 2 (l/ p t ) is not 
always an integer, Fano code can only approximate this goal. The algorithm 
for Fano coding this particular message source is illustrated in the tree graph 
of Figure 8.2. 

According to the algorithm that produces the tree graph and the Fano code it 
represents, we arrange the original symbols in order of decreasing probability, 
a (0.5), 6(0.2), c(0.2), <i(0.l), on the first leaf of the tree in Figure 8.2. Then 
we divide this ordered group into two ordered groups with as nearly equal total 
probability as possible, a (0.5) and £>( 0.2), c(0.2), and d(0.l) , on the second- 
level leaves. If the symbol is in the leftmost group, its first binary digit is 0; if 
in the rightmost group, its first binary digit is 1 . Each leaf with more than one 
symbol is again divided in the same way and the second digit is determined as 
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Table 8.2. A message source produces the symbols a, b, c, and d with given 
probabilities. Last row: the Fano binary code that represents these symbols. 


Symbol abed 


Probabilities p t 0.5 0.2 0.2 0.1 

log 2 (l /p.) 1 2.32 2.32 3.32 

Binary representation 0 10 110 111 



Figure 8.2. A tree graph that represents the Fano algorithm for binary coding of 
symbols, a, b, c, and d, that appear with probabilities 0.5, 0.2, 0.2, and 0.1. 


before. The divisions continue until each symbol is on a separate leaf. In this 
way we find the binary representations listed in the last row of Table 8.2. 

Fano code is, by design, uniquely decipherable. For example, the sequence 
111010100111000 can only be deciphered as dabbadaaa when read from left 
to right using the Fano code of Table 8.2. Some apparently more efficient ways 
of coding symbols into binary numbers are not decipherable. Consider, for 
instance, using a translation rule, a<->0,&<-»l,c<-»10, and d 1 1, shorter 
than that provided by Fano code, to decipher 111010100111000. Because we 
do not know whether the leading 1 stands by itself or is the first digit of 77, we 
do not know whether the first letter is b or d. 

Text written in the Fano code of Table 8.2 has an average of 


0.5xl + 0.2x2 + 0.2x3 + 0.1x3= 1.80 bits 


(8.17) 


8.6 Data compression and error correction 
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of Shannon entropy per transmissible symbol - a little more than the 1.76 bits 
of Shannon entropy per symbol produced by the original source. More efficient 
coding schemes exist, but they cannot compress the Shannon entropy of the 
transmissible code below the Shannon entropy of the source. 

Comparing the structure of the Shannon entropy per binary digit in Eq. 
(8.16) with the structure of the average entropy per transmissible binary digit 
in Eq. (8.17) illustrates why Fano code aims to represent the ith symbol of 
the original alphabet with a coded symbol of log 2 (l/ p l ) binary digits. If this 
were exactly possible the average Shannon entropy per character of the binary 
code would exactly match the Shannon entropy per character of the original 
alphabet. 


8.6 Data compression and error correction 

Suppose you periodically receive emails from a verbose and ne’er-do-well but 
beloved friend. After a while, you realize that the content of your friend’s emails 
always amounts to one of three messages that appear with equal frequency: (1) 
“Send money.” (2) “Can you put me up tomorrow?” and (3) “Cheers!” 

One day you receive an email from an unrecognizable source and because 
you are a technophile you know ahead of reading the message that it con- 
tains 1000 characters each one of which is coded with a standard byte, that 
is, with 8 bits. At this point the Shannon entropy of the source appears to be 
(1000 characters) x (8 bits/character) or 8000 bits. Then you look more closely 
at the message. It’s from your ne’er-do-well friend and, given its source, you 
realize, regardless of the number of its characters, the message will be one of 
the three standard ones. The entropy of the message has suddenly collapsed 
to log 2 3 or 1.58 bits - bits you can recover quickly by scanning the mes- 
sage. In effect, you have, by recognizing the redundancy of much of its text, 
compressed the entropy of the message by a factor of 8000/1.58 or 5000. Of 
course, in compressing the email some real, if relatively unimportant, informa- 
tion contained in the details of your friend’s message has been lost. Yours is a 
lossy compression. 


Lossy data compression 

Lossy data compression algorithms are especially useful in reducing audio, 
video, and multimedia files that typically store multiple gigabytes of infor- 
mation. Audio compression algorithms, for instance, eliminate acoustic fre- 
quencies to which the human ear is partially or completely insensitive. Video 
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compression algorithms eliminate bits that code parts of images that are so 
redundant their content can be accurately guessed. Typical lossy compression 
ratios of the Shannon entropy of multimedia files range from 4 to 8. 


Error correction 

However, redundant information is not all bad. Redundancy is, after all, one 
protection against the errors that accumulate in a noisy communication chan- 
nel. The best general error correction strategy is to eliminate the unimportant 
and repeat the important, that is, select compression followed by select decom- 
pression. Long before the age of digital computers Professor Will Strunk, 
author of the first draft of the bestselling text Elements of Style, had grasped 
this essential point. According to E. B. White, Professor Strunk’s favorite rule 
of style was: “Omit needless words.” Strunk 

. . . omitted so many needless words, and omitted them so forcibly and with 
such eagerness and obvious relish, he often seemed in the position of having 
shortchanged himself - a man with nothing left to say yet with time to fill, a radio 
prophet who had outdistanced the clock. Will Strunk got out of this predicament 
by a simple trick: he uttered every sentence three times .... “Rule Seventeen: Omit 
needless words! Omit needless words! Omit needless words!” 

Professor Strunk’s habit of repeating each message twice works well with 
binary code. Consider, for instance, that noise may randomly switch a binary 
digit, 1 or 0, from one to the other during transmission. If the error rate is such 
that no more than one symbol in three is switched, sending each digit three 
times will always allow the receiver to recover the original message. For if 
the original digit is 1, repeating it twice produces 1 1 1. If no more than one of 
these digits is received incorrectly, the message received will be 1 1 1 (no error), 
or Oil, 101, or 110 (one error). In any case the correct digit, 1, can always be 
recovered since it is the digit appearing most often. 

Repeating a binary digit twice in order to correct transmission errors 
increases the entropy of the received message source, relative to the entropy of 
the original message source, by a factor of 3. If the error rate is even smaller 
than one in three, we can improve on this ratio. 

Suppose, for instance, that noise changes no more than one binary digit in 
five. To minimize the required redundancy while still allowing the transmit- 
ted code to be corrected we send binary data in pairs, say. A) , Z, where X l 
and X 2 are the intended binary digits 0 or 1. Sending a pair twice produces 
Z p X 2 , X', X 2 ' where, by design, Z, = Z, and Z 2 = Z z . Suppose then that 
the received sequence Z. , Z 2 , X,' , X 2 ' is such that Z. = Z, and Z 2 £ Z 2 - the 
latter because of an error. Then the receiver would know only that either the 
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second binary digit X 2 or the fourth binary digit X 2 had been switched during 
transmission. We need to add one more binary digit to the sequence in order to 
resolve this ambiguity. In place of X,, X 2 , X.' , X 2 the coder sends the five- 
digit sequence X , , X 2 , X.' , X.' , X 3 where by design X 2 = X 2 , X 2 = X 2 , and 
X 3 is chosen so that 

X { +X 2 +X 3 = even, (8.18) 

where even stands for an even number, either 0 or 2, depending upon the values 
of X l and X 2 at the source. Then if X 2 ± X 2 and X l +X 2 +X 3 = even the receiver 
knows that X, is correct and X 2 incorrect. If, on the other hand X 2 ^ X 2 and 
X, + X 2 + X 3 = odd, where odd stands for an odd number (1 or 3), the receiver 
knows that the received value of X 2 is incorrect and X 2 is correct. Therefore, 
sending a pair of binary digits in five-digit sequences increases the entropy of 
the received code by at most a factor of 5/2 - an improvement over the factor of 
3 required by the tactic of repeating twice. Smaller error-generation rates allow 
the coder to reduce this ratio further. 


8.7 Missing information and statistical physics 

Information theory provides us with a language parallel in structure and 
vocabulary to the language with which we have described physical systems 
in Chapters 2-7. We explored part of this parallelism in Section 8.3. Here we 
continue this exploration. 

A message source is a set of symbols or messages the ith one of which 
appears in the set with a frequency that reflects its probability p y We can also 
think of a physical system, more exactly the ensemble of a physical system, 
as a message source. Imagine that we could question a physical system and at 
each query be provided with a message that describes the system microstate - 
again the ith message describing the ith microstate appearing with probability 
Pj. For this reason we can speak either of the entropy S or, equivalently, of the 
missing information, 



k (8.19) 

= ln Pj > 

of a source that produces messages that describe the ensemble of a physical 
system. The missing information described by (8.19) is measured in nats. 
However, the parallelism between information theory and statistical physics 
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goes beyond replacing one name with another. Information theory gives us a 
new way of thinking about the statistical mechanics of physical systems. 

The main task of the information theoretic approach to statistical physics is 
to assign the probabilities Pj that characterize the accessible microstates of a 
physical system. In making this assignment the information theorist is guided 
by several principles that, in effect, replace the guidance given the physicist 
by the first and second laws of thermodynamics. These principles are that the 
probabilities pj (1) quantify our knowledge of the ensemble, and (2) do so 
without introducing unwarranted assumptions or biases. Of course, (3) the 
probabilities must also be normalized so that 



( 8 . 20 ) 


where the sum is over all microstates consistent with a given macrostate or 
ensemble. 

In general we avoid unwarranted assumptions or biases by making the prob- 
abilities pj as nearly equal to each other as the constraints permit. To do so we 
maximize the missing information H subject to (8.20) and any other constraints 
that quantify our knowledge of the ensemble. Recall that missing information, 
uncertainty, and entropy are synonyms. Therefore, maximizing the missing 
information is equivalent to acknowledging all of our uncertainty. The princi- 
ple of acknowledging all of our uncertainty by maximizing the missing infor- 
mation is often called the principle of maximum entropy. 


Microcanonical ensemble 


As a simple first example, consider the case in which we know nothing about 
a physical system except that it occupies a set of microstates. We seek an unbi- 
ased assignment of the probabilities pj with which the system realizes these 
microstates. Because we want to avoid introducing bias, we maximize the 
system’s missing information H subject only to the normalization condition 
(8.20). Thus, we maximize the constrained missing information 



( 8 . 21 ) 


by making its derivatives with respect to p t vanish, and find that 
- In p t - 1 - a - 0 


( 8 . 22 ) 


and, therefore, that the microstate probabilities 



(8.23) 
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are equiprobable. Since the second derivatives of (8.21) with respect to the 
probabilities p. are negative, (8.23) identifies a relative maximum of (8.21) 
and not merely a stationary value. The result (8.23) and the normalization con- 
dition (8.20) produce p x = p 2 = •= P a = Therefore, 


H = -Y,jPj ln Pj 



= InQ, 


where fl is the number of microstates accessible to the system. 


(8.24) 


Canonical ensemble 

The energy of a system in thermal equilibrium with its environment may vary 
and in doing so realize different microstates j with probability Pj and energy E y 
In general, these microstates are occupied in rapid succession. Since the aver- 
age energy (e)|^= ^ E j J of the system is what we mean by the system’s 
internal energy, we adopt 

{ e ) = ^LjPj e j (8 - 25) 

and the normalization condition (8.20) as constraints that define the canonical 
ensemble. 

We again seek an unbiased assignment of the probabilities p { that character- 
ize this system. Thus we maximize the missing information of this ensemble 
subject to the two constraints (8.20) and (8.25). Because we have two con- 
straints we employ two Lagrange multipliers: a and fi. Maximizing the con- 
strained missing information 

“X j Pj ln Pj + a [l - X j Pj ] + P[ E ~ X j Pj E J J ( 8 - 26 ) 

means following the same procedure we have already followed twice, in 
Section 3.4 and in Section 5.4. The result is a probability distribution 



that recovers the probability p, that a system in a canonical ensemble will real- 
ize microstate i with energy E r 
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The information theoretic approach to statistical mechanics 

The information theoretic approach to statistical mechanics reproduces these 
and other standard results. But does it produce any new results? The language 
of information theory seems to cast the statistical mechanics of physical sys- 
tems into a more general framework. Certainly E. T. Jaynes makes a strong 
case that statistical mechanics is primarily an example of minimizing bias. 
The chief tool of this minimization is what he calls the principle of maximum 
entropy. (For details see his paper “Information theory and statistical mechan- 
ics” listed in Appendix VI.) But questions remain and whether or not, and if so 
in what ways, message sources and ensembles representing physical systems 
are connected in more substantial ways remains a matter of investigation. 


Problems 

8.1 Deriving the logarithm 

Show that the only continuous functions f(x) and g(x) on the domain x>0 that 
satisfy the relation (8.3), that is, n f(s) = g(s n ), are fix) = g(x) = c In x, where 
c is an arbitrary constant. (Suggested strategy: Take the partial derivatives with 
respect to n and with respect to s of the equation n f(s) = g(s n ), eliminate the 
function g(s") from these two equations, and solve for f(s).) 


8.2 Information content 

(a) A message source is composed of the following equiprobable messages: 
11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, 34, 41, 42, 43, and 44. What is the 
information content of this message source in bits? (b) A message source is 
composed of the following equiprobable messages: 11, 12, 13, 14, 21, 22, 23, 
and 24. What is the information content of this message source in bits? 

8.3 Block coding 

(a) What is the information content in bits of a message source composed of all 
possible message sequences four characters long in which each position in the 
sequence can be filled with one of 27 equally probable symbols (the 26 letters 
of the alphabet and one space)? (b) Suppose the message source described in 
part (a) is translated into another message source composed of all possible mes- 
sage sequences 12 characters long in which each position in the sequence can 
be filled with one of three equally probable symbols (the numerals 1, 2, and 3). 
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Table 8.3. A rule that translates one code into another. Used in Problem 8.3. 


a = 111 b = 112 

c = 113 

d= 121 

e = 122 

/= 123 

g = 131 

h = 132 i = 133 

/ = 211 k = 212 

1 = 213 

m = 221 

n = 222 

o = 223 

p = 231 

q = 232 r = 233 

5 = 311 t = 312 

u = 312 

v = 313 

w = 321 

x = 322 

y = 331 

Z = 332 space = 333 


(See Table 8.3 for the translation rule.) What is the information content in bits 
of this message source? (c) Show that the information content of these two 
message sources are equal. 

8.4 Information from a grade? 

What is the Shannon entropy in bits conveyed to a student who receives a grade 
(a) on the scale A, B, C, D, or F and (b) on the scale A, A-, B+, B, B-, C+, C, 
C— , D+, D, D-, and F. Assume each grade is equally probable. 


8.5 Impossible outcome 

Suppose one message contained in a source is impossible, that is, appears with 
zero probability. How much does the impossible message contribute to the 
entropy of the source? (Hint: Start with the relation 

H{p„p 2 , ...,p s , 0 ) = H{p„ Pl , ..., p s )-\im p ^ Q p\np 


and use L’Hopital’s rule to evaluate the limit.) 


8.6 Efficient Fano code 

Suppose a message source produces the letters a, b, c, and d with, respectively, 
the following probabilities: 1/2, 1/4, 1/8, and 1/8. (a) Find the Shannon entropy 
of this source, (b) Find the Fano binary code that represents these letters, (c) 
Find the average entropy per symbol of the Fano code. 

8.7 Hamming error correction 

When the error production rate of a noisy channel is no more than one binary digit 
in each sequence of seven, it is possible to send four correctable binary digits of 
data in each sequence of seven binary digits: X v X 2 , X 3 , X 4 , X 5 , X 6 , X 7 . Digits 
X V X 2 ,X 3 , and X 4 are data and the remaining three binary digits X 5 , X 6 , and X 6 
ensure correctability. These 7-digit binary sequences are called Hamming (7,4) 
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code after the computer scientist Richard Hamming (1915-1998) who invented 
this correction scheme in 1950. The correction digits X 5 , X 6 , and X 6 are deter- 
mined by the requirements 


and 


X, + X, + X 3 + X 5 = even, 
X 1 +_+_+X 6 = even, 

X x +_+_+X n = even. 


where even stands for an even number and each blank ( ) stands for either 

X v X 2 , X 3 , or X 4 . (a) Fill in the blanks in the above equations with a set of 
binary digits that make the data correctable, (b) Assuming that the binary dig- 
its 0 and 1 appear with equal probability, find the Shannon entropy of a 7-digit 
sequence of Hamming code and the Shannon entropy of the 4-digit sequence 
of binary data it represents. 


Epilogue 

What is entropy? 


While it remains true that “entropy is not a localized, microscale phenome- 
non at which we can point, even in our imaginations, and say, ‘Look! There 
is entropy’” and that, “if we insist on trying to understand a subject in ways 
inconsistent with its nature, we will be disappointed,” the eight chapters of this 
guide have prepared us to give a constructive answer to the question “What is 
entropy?” 

Any short description of entropy will necessarily be figurative. After all, one 
task of a figure of speech is to transfer a complex meaning from an extended 
description to a word or short phrase. In fact, we have already considered sev- 
eral figurative descriptions of entropy that are appropriate in special contexts: 
transformation content, disorder, uncertainty, spread in phase space, and miss- 
ing information. Transformation content was Clausius’s way of referring to 
how the entropy function indicates the direction in which an isolated system 
may evolve. Spread in phase space, while appropriate for statistical systems, 
depends upon familiarity with the technical concept of phase space. 

Disorder has long been a popular synonym for entropy. But recently order 
and disorder as describing low and high entropy systems have fallen into dis- 
favor. This is because scientists have become fascinated with isolated systems 
that generate apparent order from apparent disorder. For instance, consider a 
thoroughly shaken bottle of water and olive oil. When left undisturbed, the 
water and olive oil begin to separate into distinct layers with the less dense 
olive oil on top. Yet even in this process the entropy of the oil-water system 
increases. Thus, while order and disorder are suggestive, they can mislead. 

Missing information is a measure of the information required to choose one 
particular microstate from an ensemble of possible microstates or one message 
from a source of possible messages. For this reason missing information is 
a powerful metaphor that draws our attention to the many parallels between 
the concepts of statistical physics and information theory. The concept of 
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uncertainty plays a similar role. The larger and more varied the ensemble from 
which a system can be chosen and the larger and more varied the message 
source from which a message can be drawn, the more uncertain the outcome 
and the higher the entropy. 

Entropy as possibility is my favorite short description of entropy because 
possibility is an apt word and, unlike uncertainty and missing information, has 
positive connotation. Thus, according to the second law of thermodynamics, 
an isolated thermodynamic system always evolves in the direction of opening 
up new possibilities. And the larger the set of possibilities from which either a 
microstate or a message can he realized or chosen, the larger the entropy of the 
physical system or the Shannon entropy of the message source. 

A formal definition of entropy in terms of possibility is: Entropy is an addi- 
tive measure of the number of possibilities available to a system. Thus the 
entropy of a physical system is an additive measure of the number of possible 
microstates that can be realized by that system. And the entropy of a message 
source is an additive measure of the number of possible messages that can be 
chosen from that message source. The image on the cover of this book almost 
perfectly captures the sense of entropy as possibility. As the constraints that 
inform a living organism dissolve, the entropy of the organism increases. As 
the flower dies, its seeds are scattered in the breeze. Yet even in this death, new 
possibilities are sown. 



Appendix I 

Physical constants and 
standard definitions 


Physical constants in SI units 


Name 

Symbol and value 

Boltzmann constant 

k = 1 .38 x 1 (T 23 m 2 kg/(s 2 K) 

Planck constant 

h = 6.63 x 10” 34 m 2 kg/s 

Gas constant 

R = 8.3 1J/(K mole) 

Electron mass 

=9.11x10 31 kg 

Electron charge 

e = 1.60x10 19 C 

Standard acceleration of gravity 

g = 9.81 m/s 2 

Speed of light in vacuum 

c = 3.00 x 10 s m/s 

Radiation constant 

a = 7.57 x 10- 16 kg/(m s 2 K 4 ) 

Atomic mass unit 

u = 1.66 x 10- 27 kg 

Standard definitions 

Name 

Symbol and definition 

Mechanical equivalent of heat 

7 = 4.19 J/cal 

Atmospheric pressure 

1 atm 1.01 x 10 5 Pa 


= 760 torr 


= 760 mm Hg 


= 14.7 lbs/in 2 

Bar 

1 bar = 10 5 Pa 

Electronvolt 

1 eV = 1.60 x 10- 19 J 
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Formulary 


ln(xy) = lnx + lny 
In — = lnx- lny 

w 

lny 

r» x 3 dx _ K* 

Jo e x -l~l5 

j;4^ = 2Z(3) « 2.404 

J^e'^dx = yfn 


\n{l + x) = x~- 
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Absolute entropy: The unique value of entropy that describes the macrostate of a system. 
Same as Third law entropy. 

Absolute temperature: A temperature determined by the efficiency of a Carnot engine 
operating between a heat reservoir in a standard state and a reservoir in thermal 
equilibrium with the object whose temperature is to be determined. Same as 
Thermodynamic temperature. 

Additivity: A thermodynamic variable is additive when its value is the sum of the 
corresponding values describing each part of a composite system. The entropy 
S and energy E of a thermodynamic system are additive. The component parts to 
which additivity applies are sometimes but not always spatially separate. Related 
to Extensive variable. 

Average energy approximation: A description of the ideal quantum gases in terms of 
the number of microstates V (E/N) 2/2 ( Amm / 3 h 2 ) / available to a single particle 
that occupies volume V and possesses the average energy E/N of a system of N 
particles. Here e stands for the base of the natural logarithm, m for the mass of the 
particle, and h for Planck’s constant. 

Bandgap energy: The separation in energy between the valence and conduction band 
electrons in a semiconductor. 

Blackbody radiation: Electromagnetic radiation in equilibrium with a material system 
at thermodynamic temperature T. The thermodynamic variables of a system of 
blackbody radiation are its energy E and its volume V. Also known as equilibrium 
radiation. 

Boltzmann distribution: The distribution of microstate probabilities p t = e^‘ .e^ 

for microstate energy E t and where /i = \/kT appropriate for a system that is in 
thermal equilibrium with its environment at temperature T. When the system is a 
single particle of an ideal gas the Boltzmann distribution is a Maxwell-Boltzmann 
distribution. Same as Canonical distribution. 

Boltzmann entropy: The third law compliant entropy of the macrostate of a system 
described by S = fcln£2 where fl is the number of equally probable microstates of 
the macrostate of an isolated system. 
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Bose-Einstein condensate: A low temperature phase of matter composed of bosons, that 
is, particles that obey Bose-Einstein statistics. At sufficiently low temperatures, 
bosonic matter condenses into a macrostate in which all its particles occupy one 
single-particle microstate. 

Bosons: Particles that obey Bose-Einstein statistics. According to Bose-Einstein 
statistics, any number of identical bosons can occupy the same single-particle 
microstate. 

Canonical distribution: Same as Boltzmann distribution. 

Canonical ensemble: A collection of identical systems that individually realize 
their microstates with probabilities p, jj* e~ E ‘^ kT J characteristic of a system in 
equilibrium with an environment at temperature T. 

Carnot efficiency: The maximum efficiency of a heat engine that operates between two 
heat reservoirs. The Carnot efficiency is given by e Carnot = 1 where l v is 

the temperature of the colder temperature reservoir and T H is the temperature of 
the hotter reservoir. 

Carnot’s theorem: Asserts that the most efficient heat engine operating between a pair 
of heat reservoirs is one that operates reversibly. 

Chemical potential: The intensive variable brought into existence by the variability 
of the number of particles that compose a system. The chemical potential is the 
increment in energy when a fluid system grows by one particle given that its 
entropy and volume remain constant. Symbolically, the chemical potential p is 
described by p = ( dE/dN) s v or by p = -T(dS/dN) v f : 

Classical microstate: The description of a system that consists of identifying the 
location of the system’s distinguishable particles within arbitrarily constructed 
phase space cells. 

Classically distinguishable particles: Particles that can, in principle, always be 
distinguished from one another because of their distinct trajectories in space 

Complementarity: The principle that quite different, even contradictory, physical 
models can describe a single phenomenon. The wave and photon pictures describe 
electromagnetic radiation in complementary fashion. 

Compton effect: The scattering of an X-ray photon from a free or essentially free 
electron. In the process the X-ray photon transfers some momentum and energy 
to the electron. 

Correspondence principle: Quantum expressions reduce to classical ones in an 
appropriate limit. 

Degenerate, Degeneracy: Words that literally mean “not normal.” In statistical 
mechanics these words refer to system or single-particle microstates having the 
same energy. Thus, two particles are degenerate when they occupy the same single- 
particle microstate or different single-particle microstates with the same energy. 
Less frequently, the multiplicity SI of an isolated system is called its degeneracy 
since all the system’s microstates have the same energy. Also, a ground macrostate, 
that is, a T = 0 macrostate, is degenerate when it is composed of more than one 
microstate. 

Efficiency: The efficiency of a heat engine operating between two heat reservoirs is the 
ratio of the work performed by the engine in one cycle W to the heat extracted from 
the hotter reservoir in one cycle Q H , that is, W/Q# ■ 
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Einstein solid: A crystalline array of atoms or molecules each one of which oscillates 
simple harmonically and independently in three dimensions with a common 
frequency. The energy of an oscillator in an Einstein solid is quantized. 

Einstein temperature: The parameter hv o jk where the symbol v o is the common 
frequency with which all the atoms of an Einstein solid vibrate. The Einstein 
temperature was used by Einstein to fit the specific heat of an Einstein solid to 
available data. 

Ensemble: A large number of fictional copies of a system. The frequency with which the 
copies in an ensemble are found in a particular microstate reflect the probability 
of that microstate. Some standard ensembles have been given special names. The 
microcanonical ensemble, for instance, contains copies of a completely isolated 
system with a given energy, and the canonical ensemble contains copies of a 
system in thermal equilibrium with an environment at a given temperature T. 

Entropy: A measure of the irreversibility of the thermodynamic evolution of an isolated 
system. Also, a measure of the spread in phase space , the uncertainty, or the missing 
information of the macrostate of a system. An additive measure of the number of 
configurations available to a system, or, equivalently, an additive measure of the 
number of possibilities open to a system. 

Equation of state: A relation among the thermodynamic variables that describe a system 
macrostate. 

Equilibrium state: A time-independent macrostate of a thermodynamic system. Same 
as Thermodynamic state. 

Equipartition theorem: Asserts that every term in a particle’s energy that is quadratic in a 
phase space coordinate adds kT/ 2 to the system’s internal energy. The equipartition 
theorem is observed in classical systems when the phase space integrations of the 
partition function can be extended indefinitely. 

Extensive variable: A thermodynamic variable is extensive when its value is the sum of 
the values describing each part of a homogeneous composite system. Extensivity is 
a special case of Additivity. For example, the energy E, volume V, particle number 
N, and entropy 5 of a simple fluid are extensive variables. 

Fano code: A simple method of translating an alphabet of symbols into binary digits that 
aims to keep the entropy of the binary digits as close as possible to the Shannon 
entropy of the original alphabet. 

First law of thermodynamics: The law of conservation of energy applied to a 
thermodynamic system. 

Fluid: Any thermodynamic system that can be described with only two state variables: 
an isotropic pressure P and volume V. Same as Simple fluid. 

Fundamental constraint (of a fluid system): The relation dE = TdS - PdV among the 
differentials of the thermodynamic variables, energy E, temperature T, entropy S, 
pressure P, and volume V of a fluid system. When a fluid system is not closed, its 
fundamental constraint becomes dE = T dS - P dV + pdN where p is the chemical 
potential and N is the number of particles composing the system. 

Fundamental postulate ( of statistical mechanics ): Asserts that all microstates composing 
the macrostate of an isolated system are equally probable. 

Gibbs entropy formula: The relation S = —k£ Pj In pj that determines the entropy S 
from the probabilities Pj [j = 1, 2, . . .J that a system will occupy its y'th microstate. 
The entropy determined from the Gibbs entropy formula is third law compliant. 
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Ground state: The macrostate of a system in the T — > 0 limit, that is, with the lowest 
possible energy. 

Ground state pressure: The pressure that persists in the T — > 0 limit of an ideal Fermi 
gas. Sometimes called the degeneracy pressure. 

Ground state energy: The energy that persists in the T — » 0 limit of an ideal Fermi gas. 
Sometimes called the degeneracy energy. 

Heat: Energy transferred by heating or cooling. 

Heat engine: A device that uses a temperature difference to produce work. 

Heat reservoir: Same as Temperature reservoir. 

Hole: The vacancy left in the valence band of a semiconductor when an electron is 
excited from the valence band into the conduction band. Both valence band holes 
and conduction band electrons contribute to the conductivity of a semiconductor. 

Information: Ralph Flartley’s word for the information content of a message source 
composed of equally probable symbols or messages. Information is the logarithm 
of the number of equally probable symbols or messages in a message source. 

Intensive variable: A thermodynamic variable is intensive when its value remains 
constant as the system is scaled up or down. Temperature T, pressure P, and 
chemical potential p are intensive fluid variables. 

Internal energy: The energy E contained in a thermodynamic system. The internal 
energy excludes the energy associated with the motion or the position of the 
system as a whole. 

Intrinsic semiconductor: A crystalline semiconductor with no defects or impurities. 

Irreversible process: A thermodynamic process whose direction can be reversed only 
by making finite changes in the state of the system or its environment. A process 
that is not quasistatic or one that proceeds with friction, dissipation, or hysteresis 
is irreversible. See also Reversible process. 

Loschmidt’s paradox: All thermodynamic systems are composed of fundamental 
particles, all thermodynamic processes are composed of the interactions of 
fundamental particles, and all interactions of fundamental particles are reversible; 
but all non-idealized thermodynamic processes are irreversible. 

Lossy compression: The reduction of the Shannon entropy of a message source by 
eliminating redundant or inessential information. 

Macrostate: A set of microstates that can be described with a small set of thermodynamic 
variables. 

Maxwell’s demon: A small being, or inanimate automaton, that can decrease the entropy 
of a system by exploiting the fluctuations in its parts. 

Message source: The set of all possible messages from which particular messages can 
be drawn. An ensemble of messages. 

Microcanonical ensemble: An ensemble composed of copies of a completely isolated 
system with a given energy E. 

Microstate: The most precise possible description of the arrangement of the atoms and 
molecules of a system. See also Classical microstate. 

Missing information: The information needed to specify the microstate of a system 
given its macrostate or the information needed to select a particular message 
from a message source. Same as Shannon entropy. Also the third law entropy of a 
physical system divided by Boltzmann’s constant k. 
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Multiplicity: The number of microstates that an isolated system can occupy. Sometimes 
called the Degeneracy or the Thermodynamic probability or the Statistical weight 
of a system. While the traditional symbol for multiplicity is Cl, the symbol for 
multiplicity that appears on Boltzmann’s tombstone is W. 

Occupancy: The number of particles in a multi-particle system divided by the number of 
accessible single-particle microstates. The average occupation number of a system. 

Occupation number (of a cell): The number of particles of a multi-particle system that 
occupy the same cell in a 6-dimensional phase space common to all the particles 
of the system. 

Partition function: The quantity Z [=X, e w ^ ere the sum is over all the 

microstates of the system and E t is the energy of the system when in microstate i. 
Compare with Single-particle partition function. 

Pauli principle: Asserts that no two atomic electrons can occupy the same single- 
particle microstate. 

Phase space: A multi-dimensional space whose coordinates are the position and 
momenta of the particles composing the system. 

Phonons: Bundles of energy whose size is determined by the units of energy that can be 
absorbed or rejected by the oscillators that compose a solid. 

Photoelectric effect: High frequency light shines on a metallic surface and ejects 
electrons. 

Photon: A quantum of light. The energy s and frequency v of a photon are related by 
E = hv while its momentum p and frequency v are related by p = hv/c. 

Quantum indistinguishability: Two particles are quantum mechanically indistinguishable 
when exchanging them would not lead to experimentally observable results. If two 
particles are quantum mechanically indistinguishable, they are indistinguishable 
for the purpose of counting microstates. 

Quasistatic process: An indefinitely slow process. A system experiencing a quasistatic 
process passes through a continuum of thermodynamic states. 

Radiation constant: The proportionality constant between the energy density E/V 
of blackbody radiation and the fourth power of its absolute temperature 7' 4 . 
The radiation constant is usually denoted with the symbol a. Thus, E/V = aT 4 . 
Quantum statistical arguments show that a = ^jfk^ji^Sc^h^. 

Random variable: A variable that realizes a range of values according to a set of 
probabilities. 

Reversibility paradox: Same as Loschmidt’s paradox. 

Reversible process: A thermodynamic process whose direction can be reversed by 
making only an indefinitely small change in the system or in its environment. A 
reversible process necessarily proceeds indefinitely slowly, that is, quasistatically, 
and without friction, dissipation, or hysteresis. See Irreversible process. 

Semi-classical limit: The limit of a quantum statistical expression in which kT is much 
greater than a single quantum of energy. 

Shannon entropy: A measure of the information content of amessage source. The Shannon 
entropy of a source of one-character messages is given by H = —c£ p t In p, where 
p i is the probability that the character is realized by the ith symbol and c is a 
constant that determines the unit of information. 


168 


Appendix III Glossary 


Simple fluid: Same as Fluid. 

Single-particle partition function: The quantity Z, I = ^ where the sum 

is over all the microstates of a single-particle in a phase space common to all 
particles and where Ej is the energy of the single particle in microstate j. Single- 
particle partition functions are useful whenever the particles of a system occupy 
their positions in a common phase space independently of each other except for 
system-wide constraints such as those imposed on the total energy E and particle 
number N. 

Spin-statistics theorem: Asserts that particles with even units of spin, called bosons, can 
occupy the same single-particle microstate and particles with odd units of spin, 
called fermions, cannot occupy the same single-particle microstate. Wolfgang 
Pauli (1900-1958) discovered the spin-statistics theorem in 1940. 

Statistical weight: See Multiplicity. 

Stigler’s law of eponymy: Asserts that no discovery is named after its original 
discoverer. 

Sure variable: A variable that realizes a single, definite value. Not a Random variable. 

Temperature, empirical: Hotness measured on a scale defined by the behavior of a 
physical system, that is, as determined by a Thermometer. 

Temperature reservoir: A system that maintains the same temperature regardless of how 
much heat the system absorbs or rejects. A temperature reservoir has indefinitely 
large heat capacity. Same as Heat reservoir. 

Thermodynamic probability: Max Planck’s term for Multiplicity. Other synonyms are 
thermodynamic weight and statistical weight. Also equivalent to the Degeneracy 
of an isolated system. 

Thermodynamic system: A macroscopic system that observes the fundamental postulate 
of statistical mechanics. A thermodynamic system can be described with a small 
set of thermodynamic variables. 

Thermodynamic state: Same as Equilibrium state. 

Thermodynamic temperature: Same as Absolute temperature. 

Thermodynamic weight: Same as Multiplicity. 

Thermometer: A physical system that associates the size of a convenient thermodynamic 
variable, such as volume, resistance, or color, with a unique number. The way in 
which the thermodynamic variable is associated with a unique number defines a 
temperature scale. 

Third law entropy: A formulation of the entropy that exploits our freedom to adopt 
a conventional value for the entropy in the limit of vanishing thermodynamic 
temperature. The formulations S = Gn£2 and 5 = p f In p i are third law 
entropies. Same as Absolute entropy. 
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Anders Celsius (1701-1744) devises an empirical temperature 
scale that assigns 0 to the boiling point of water and 100 to its 
freezing point. 

Carolus Linnaeus (1701-1778) turns Celsius’s scale upside down. 

John Dalton (1766-1844) enunciates a “law of simple multiple propor- 
tions” that summarizes the way different elements combine and sug- 
gests the existence of atoms and molecules. 

The law of Dulong and Petit is discovered. According to this law the 
molar specific heat capacity C v /n of a solid is a universal constant 3 R 
0 25 J /(Kmole)] where R is the so-called gas constant. 

Sadi Carnot (1796-1832) publishes On the Motive Power of Fire 
in which he articulates a version of the second law and proves 
Carnot’s theorem. 

James Joule’s precision experiments compel acceptance of the first law 
of thermodynamics. 

William Thomson (1824-1907) (later known as Lord Kelvin) invents 
the concept of absolute or thermodynamic temperature. 

Rudolph Clausius (1822-1888) articulates a version of the sec- 
ond law and distinguishes between the first and second laws of 
thermodynamics. 

William Thomson (Lord Kelvin) articulates a version of the 
second law. 

Rudolph Clausius discovers the concept of entropy. 

James Clerk Maxwell (1831-1879) publicly introduces Maxwell’s 
demon in his book Theory of Pleat. Maxwell’s purpose is to demon- 
strate that entropy is a statistical or probabilistic quantity. 

Johann Loschmidt (1821-1895) asks, “Why, if the interactions of ele- 
mentary particles are reversible, are thermodynamic processes irrevers- 
ible?” This disjunct between microscopic and macroscopic physics is 
known as Loschmidt’s paradox. 
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1877 Ludwig Boltzmann (1844—1906) publishes a paper On the relation of the 
second law of the mechanical theory of heat and the probability calculus with 
respect to the theorems on thermal equilibrium in which he first introduces 
the relation of entropy to macrostate multiplicity and identifies the most prob- 
able occupation number macrostate with the state of thermal equilibrium. 

1900 Max Planck (1858-1947) announces a derivation of the spectral energy den- 
sity of blackbody radiation in which he is forced to assume that light is emit- 
ted from matter in discrete, quantized amounts. 

1902 J. Willard Gibbs (1839-1903) publishes Elementary Principles of Statistical 
Mechanics. This book presents the major methods and applications of classi- 
cal statistical mechanics and introduces the concept of an ensemble. 

1905 Albert Einstein (1879-1955) publishes groundbreaking papers on Brownian 
motion, special relativity, and the photoelectric effect. 

1906 Walther Nemst (1864—1941) formulates what has since became known as the 
third law of thermodynamics. 

1907 Einstein proposes a simple quantum model of a crystalline solid, the so-called 
Einstein solid, which initiates the modem study of condensed phase physics. 

1912 The Sackur-Tetrode equation for the extensive entropy of an ideal gas with 
quantized phase space is introduced. 

1923 Arthur Holly Compton (1892-1962) observes the Compton effect in which 
a photon striking a free electron transfers energy and momentum to the 
electron. 

1924 July. S. N. Bose (1894-1974) derives the spectral energy density of black- 
body radiation by counting the microstates of a gas of photons. 

1924 July. Einstein applies Bose’s method of counting microstates to a gas of par- 
ticles with non-zero rest mass. 

1925 January. Wolfgang Pauli (1900-1958) proposes the Pauli principle according 
to which no two electrons within an atom can share the same single-particle 
microstate. 

1925 January. Albert Einstein predicts the existence of a new phase of matter later 
called the Bose-Einstein condensate. 

1925 July. Wemer Heisenberg (1901-1976) formulates the first version of the new 
quantum mechanics. 

1926 January. Erwin Schrodinger (1887-1961) introduces the wave mechanics ver- 
sion of quantum mechanics. 

1926 February. Enrico Fermi (1901-1954) describes the statistics of particles no 
two of which can share the same single-particle microstate, particles later 
known as fermions. 

1926 August. P. A. M. Dirac (1902-1984) relates symmetric and antisymmetric 
wave functions with, respectively, Bose-Einstein and Fermi-Dirac statistics. 

1928 R. V. L. Hartley (1988-1970) defines a measure H [oc Ins"] of the informa- 
tion content of a message source that produces messages of length n each 
character of which can realize s equally probable symbols. 

1929 Leo Szilard (1898-1964) analyzes the interaction of Maxwell’s demon with a 
one-particle gas. Szilard’s paper initiates a line of research, active to this day, 
on the connections between information and entropy. 
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1933 Ludwig Boltzmann’s tombstone in the Vienna Central Cemetery, inscribed 
with the equation S = klogW , is dedicated. 

1940 Wolfgang Pauli discovers the spin-statistics theorem according to which par- 
ticles with even units of intrinsic spin (bosons) can share the same single- 
particle microstate while particles with odd units of intrinsic spin (fermions) 
cannot share the same single-particle microstate. 

1948 Claude Shannon (1916-2001) initiates the modem study of informa- 
tion theory with the publication of his paper The Mathematical Theory of 
Communication. 

1949 Robert Fano (1917—) and, independently, Claude Shannon invent a coding 
scheme called Fano or Shannon-Fano coding according to which an alphabet 
of symbols can be efficiently translated into binary digits. 

1950 Edward Purcell (1912-1997) and Robert Pound (1919-2010) create a system 
of nuclear spins that exhibits negative temperature. 

1995 Eric Cornell (1961—) and Carl Wieman (195 1-) cool a gas of weakly interact- 
ing bosons (rubidium atoms) below the critical temperature and produce a 
Bose-Einstein condensate. 
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1.1 The entropy of the fluid system increases because the composite weight-fluid sys- 
tem is an isolated system that suffers an irreversible change. 

1.3 Cln(r / /7;.). 

1.4 A5 = n/fln(v r / /V;). 

1.5 AS/n = -(fl/2)ln2. 

1.6 (a) Equations of state (3) and (4) do not observe the first and second laws 

of thermodynamics. Equations of state (1), (2), and (5) do observe the first and 
second laws of thermodynamics. 

(b) The entropy function for equations of state ( 1) is S (E, V) = a In V - b 2 /E , for (2) 
is S(E,V) = bln[EV), and for (5) is S(E,V) = abV + Mn£. 

1.7 T = l/aV 1 and E = laEVT . 

1.8 (a) E = C v T + (V- E o ) 2 /( 2k To V 0 ) and P = a Po T/K To -(V- V 0 )/(k t X e „) ■ 

(b) No. As T — ► 0, S does not go to a finite constant independent of the thermody- 
namic state variables. 

1.9 (1) violates the third law, (2) violates the third law, (3) violates the third law and 

violates T > 0, (4) violates the third law, (5) violates the third law, (6) violates 
the third law and violates T> 0, and (7) violates neither requirement. 

2.1 (a) AS = feln(3 1100 /2 1000 ). 

(b) (2/3) 1000 (1/3) 100 . 

2.2 (a) 4, (b) 13, (c) 52, and (d) these two events are independent. 

2.3 14%, about n = 90. 

2.4 (a) n N and lO™. 

(b) [N + n - 1 )\/N\(n - 1)! and 1.34 x 10 40 . 

(c) n\/N\(n - N)\ and 1.01 ■ 10 29 . 

2.5 (a) £2 = A !/[re + ! n_ !j. 

(b) S(L) = c + (M/2){21n2 - (l - L/Na) ln(l - L/Na) - (l + L/Na) ln(l + L/Na)}. 

(c) F = (kTI 2a)[ln(l - L/Na) - ln(l + L/Naj] . 

(d) F = -(AT / Na 2 )L. 
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3.1 2.26 xlO 19 . 

3.2 (a) E = 3NkT/2 - aN 2 /v. 

(b) ${E, V t N) ~ c (N) + Nk [in (V - Nb) + (3/2) ln(£ + N 2 a/v) 

+ (3/2)ln(4roe/3 Nm)]. 

(d) P = NkT/[v - Nb) - aN 2 /v 2 . 

3.3 (a) E = 5NkT/2 and PV = NkT . 

(b) Each molecule has five degrees of freedom - three translational and two ro- 
tational. 

3.4 AS = (2PV/T)ln2. 

3.5 S(E,N) = pN + 3iVfc[l + ln(E(3NHv B )j where p is any real number. 

3.6 517 m/s 2 . 

4.1 (a) hvJkT = In[(x + Vi)/{x - Vi)\ and Z t = -'/^{x + Vt) . 

4.2 6-lxlO -6 . 

4.4 (a)Z s =1+e ° /kT . 

(b) E = Nee- elw l{\ + e E ' kT ). 

(.c)C = Nk(e/kTf/[l + e e,kT ) 2 . 

(d) When the temperature is sufficiently low the heat capacity is also low because 
the system cannot absorb any energy unless it absorbs enough to start popu- 
lating the high energy state. Alternatively, when the temperature is too high, 
the heat capacity is low because most of the particles are already in the high 
energy state. 

4.7 p/T = 5k/2 - S/N where S is given by the Sackur-Tetrode entropy (4.2). 

4.8 (a) n + /N = e 2m ’ BjkT /[l + e 2m " BjkT ) and n_/N =l/(l + e 1 "' 1,Bjkr ). 

(b) lim r > _ n^/N = 1/2 , lim r ^ 0 n + /N = 1 , lim r ^ 0 n_/N = 1/2 , and 
li m r-> 0 n _/N = 0 . 

(c) 0.122 K. 

4.9 (a) C = Nkx 2 e 2 *j (l + e*f where x = 2m B B 0 /kT. 
(b)C->0asr->°°,r->-~ a ndr->0. 

4. 10 (a) M = Nm B [( e 2m ’ BjkT - 1 )/ ( e 2m -- BJkT + 1)]. 

(c )C = Nm 2 B /k. 

5.1 (a) Occupied l/(l + e~^ w ) and unoccupied e ^ kT j{\ + e e/kT '). 

(b) S = k^s/k^e ^K 1 + e**) + ln(l + e^* r )}. 

6.2 (b) [N/E) 3l2 (N/v){h 2 /2mf 3 >1. 

(c) N/n > (3/2 roe) 3/2 = 0.0742. 

6.4 NkT /E o ~ 0.377. Therefore, the gas is in the quantum regime. 
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6.5 K = )(5/27k)(3/4tt) 2/3 . 

7.1 (a) P v = -(SnkTv 2 dv/c 2 ) ln(l - e h ' ,lkT ). 

(b) P = (8tr s /45)(fe 4 r 4 //iV), 

7.2 4.04 x 10 13 Pa. The ratio of radiation to gas pressure at the center of the Sun is 
1.00x10 3 . 

7.3 (a) e(A) = \$nh/ cXf j[e hclm - 1) . 

(bU mai =0.2014(fcc/fcr). 

(c) 5800 K. 

7.4 60.4 V(kT/chf . 

7.5 E = 1.27 NkT{T/T c f' 2 . 

8.2 (a) 21og 2 4 = log 2 16 or 4 bits. 

(b) log 2 8 or 3 hits since here the messages rather than the symbols out of which 
they are composed are equiprobable. 

8.3 (a) 41og 2 27 or 19.02 bits. 

(b) 121og 2 3 or 19.02 bits. 

8.4 (a) log 2 5 or 2.32 bits. 

(b) log 2 12 or 3.58 bits. 

8.6 (a) 1 .75 bits, (b) a 0, b <-> 10, c 1 10, and d 1 1 1. (c) 1.75 bits per symbol. 

8.7 (a) One set of blanks with X 2 and X 4 and the other with X 3 and X 4 . (b) 7 bits 

and 4 bits. 
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An item appears here for at least one, and sometimes for both, of two reasons: I con- 
sulted it in writing this guide and I recommend it for further study. Publication dates 
always refer to the first edition, unless two dates appear. Then the earlier date refers to 
the first edition and the later date to the edition actually consulted. 

von Baeyer, Hans Christian. Maxwell’s Demon: Why Warmth Disperses and Time Passes. 
207 pages. Random House, New York, 1998. An engaging, popular presentation 
of thermodynamics, statistical and quantum mechanics. Maxwell’s demon, and 
information theory. 

von Baeyer, Hans Christian. Information: The New Language of Science. 272 pages. 
Harvard University Press, Cambridge, Massachusetts, 2005. Another helpful, 
popular book by Hans von Baeyer. 

Baierlein, Ralph. Atoms and Information Theory. 486 pages. W. H. Freeman, San 
Francisco, California, 1971. A thorough and reliable statistical mechanics text 
framed in the language of information theory. 

Baierlein, Ralph. Thermal Physics. 442 pages. Cambridge University Press, Cambridge, 
UK, 1999. Another quite helpful Baierlein text. The “thermal physics” approach 
presents thermodynamics and statistical mechanics as one subject. 

Boltzmann, Ludwig. Lectures on Gas Theory. Translated by Stephen Brush. 490 pages. 
New York, Dover, 1995 (1896 and 1898). Boltzmann’s writing is neither concise 
nor clear. His derivation of the Maxwell distribution from the hypothesis of equally 
probable microstates appears on pages 55-58. 

Carnot, Sadi and others. Reflections on the Motive Power of Fire And Other Papers 
on the Second Law of Thermodynamics. Dover, New York, 2005. Contains the 
title essay as well as historically important essays by Emile Clapeyron and Rudolf 
Clausius. 

Cover, Thomas M. and Joy Thomas. Elements oflnformation Theory, 2nd edition. 776 
pages. Wiley, Hoboken, New Jersey, 2006. A standard information theory text with 
a theorem-proof presentation of the subject. Aimed at upper level undergraduate 
and graduate students. 

Feynman, Richard R, Robert B. Leighton and Mathew Sands. The Feynman Lectures on 
Physics, Vol. 3. Addison- Wesley, Reading, Massachusetts, 1965. Section 3-4 has 
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a nice discussion of the way the superposition of the wave functions of identical 
particles generates the distinction between bosons and fermions. 

Gibbs, J. Willard. Elementary Principles in Statistical Mechanics. 207 pages. Ox Bow 
Press, Woodbridge, Connecticut, 1981 (1902). A masterful if difficult presentation 
of classical statistical mechanics by one of its originators. 

Glieck, James. The Information. 526 pages. Pantheon, New York, 201 1. An expansive, 
popular history of information from cuneiform to cyberspace. 

Gould, Harvey and Jan Tobochnik. Statistical and Thermal Physics. 511 pages. 
Princeton, Princeton, New Jersey, 2010. A thermal physics text with fully integrated 
computational examples and exercises. 

Jackson, E. Atlee. Equilibrium Statistical Mechanics. 242 pages. Dover Publications, 
New York, 2000 (1968). A compact introduction to statistical mechanics. Jackson’s 
short appendix “Concerning the Entropy Constant” is noteworthy. 

Jaynes, E. T. Information theory and statistical mechanics. Physical Review, Volume 
166, pp. 620-630, May 15, 1957. Jaynes makes a strong case for the “maximum- 
entropy” information-theoretic approach to statistical mechanics. 

Hartley, R. V. L. Transmission of information. Bell System Technical Journal, Volume 
7, Number 3, pp. 535-563, July 1928. A precursor of Shannon’s 1948 paper on 
information theory. 

Kuhn, Thomas. Black-Body Theory and the Quantum Discontinuity 1894-1912. 378 
pages. University of Chicago Press, Chicago, Illinois, 1978. History of science at 
its best. Kuhn argues that in 1900 Planck did not realize he had quantized anything. 
In the eight years following 1900 others, notably Albert Einstein, forced Planck to 
recognize the revolutionary nature of his own work. 

Leff, Harvey and Andrew F. Rex. Maxwell’s Demon 2: Entropy, Classical and Quantum 
Information, Computing. 485 pages. Institute of Physics, Bristol, U.K., 2003. An 
anthology of papers inspired by Maxwell’s imagined entropy-reducing beast that 
search out possible connections, formal and substantial, between the entropy of 
physics and chemistry and the entropy of information theory. Complete with an 
introductory essay and a collection of demon visuals. 

Lemons, Don S. Mere Thermodynamics. 207 pages. Johns Hopkins, Baltimore, 
Maryland, 2008. A presentation of thermodynamic laws, methods, and applications 
that emphasizes the logical structure of thermodynamics. 

Lindley, David. Boltzmann’s Atom. 260 pages. Simon & Schuster, New York, 2001. A 
delightful non-technical biography of Boltzmann that skillfully weaves together 
the content and context of Boltzmann’s science. 

Pais, Abraham. Subtle is the Lord .... 552 pages. Oxford University Press, Oxford, UK, 
1982. A biography of Einstein that contains detailed explanations of his physics 
and the physics of his era. Some formal background in physics is required to fully 
appreciate this biography. 

Penrose, Oliver. Foundations of Statistical Mechanics: A Deductive Treatment. 272 
pages. Dover, New York, 2005 (1970). Quite valuable but at a considerably higher 
level and density of mathematics than this guide. 

Planck, Max. The Theory of Heat Radiation. 224 pages. Dover, New York, 1991 (1914). 
Planck absorbed Boltzmann’s ideas in order to articulate the statistical mechanics 
of blackbody radiation. Admirably clear if also tedious in detail and somewhat 
dated. Influenced the presentation of ideas in Chapters 2 and 4 of this Guide. 
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Schroeder, Daniel V. Thermal Physics. 422 pages. Addison Wesley, Upper Saddle, New 
Jersey, 1999. A deservedly popular thermal physics textbook. 

Shannon, Claude and Warren Weaver. The Mathematical Theory of Communication. 
125 pages. University of Illinois Press, Urbana, Illinois, 1964 (1949). Contains the 
1948 paper that initiated information theory. Shannon’s paper is fairly mathematical 
relative to most of the items in this bibliography. However, Shannon also motivates 
and explains. Weaver’s short expository article on information theory originally 
appeared in Scientific American. 

Singh, Jagjit. Great Ideas in Information Theory, Language and Cybernetics. 338 
pages. Dover, New York, 1966. The first seven chapters (83 pages) are a nice non- 
technical introduction to information theory. 


Index 


absolute temperature, 15-17 
additivity of entropy, 9, 10, 40-41, 50 
of Shannon entropy, 146 
average energy approximation, 110, 

118-119, 127, 137 
applied to ideal Bose gas, 127 
applied to ideal Fermi gas, 1 12 
and ideal quantum gases, 137 

blackbody radiation, see Wiens law and Stefan 
Boltzmann law 
entropy, 60 
equations of state, 24 
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and Brownian motion, 33 
and photon, 122 
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ensemble, 97 
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and Gibbs entropy formula, 96-98 
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microcanonical, 103, 154 
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of Shottky defect, 87 
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of ideal Bose gas, 129 
of ideal Fermi gas, 114 
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of intrinsic semiconductor, 109 
of paramagnet, 90 
of Shottky defect, 88 


equipartition theorem, 69-70 
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of ideal Fermi gas, 120 
of ideal gas, 58 
of ideal solid, 74 
of Shannon entropy, 146 

Fano code, 148-151, 157 
Fermi, Enrico, 106 
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fermions, 106, 108, 111, 118 
first law of thermodynamics, 2, 19 
fluid, simple, 7, 22 

blackbody radiation as, 24 
fundamental constraint of, 23 
fundamental postulate of statistical mechanics, 
36, 37, 96 

Gibbs entropy formula, 96-98, 99, 

100, 103, 104 
Gibbs free energy, 20 
and stability, 20 
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Gibbs, J. Willard, 97 
ground state 

and third law of thermodynamics, 86 
of Einstein solid, 82 
of ideal Bose gas, 130, 132, 133-136 
of ideal Fermi gas, 115, 117 
ground state pressure, 118 
of conduction electrons, 119 
of electrons in white dwarf stars, 121 
of ideal Fermi gas, 118 
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Hamming, Richard, 158 
harmonic oscillator 
classical, 67 
quantized, 76-78 

single particle partition function of, 79 
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Hartley’s information content, 142, 147 
heat, 2, 14, 17, 22 
used as a noun, 9 
heat capacity, 29 
of Einstein solid, 81 
of fluid 

at constant volume, 116 
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heat capacity ( cont .) 
of ideal Bose gas, 131 
of ideal Bose gas and condensate, 139 
of ideal classical solid, 69 
of ideal Fermi gas, 116, 120 
of ideal gas 
at constant volume, 24 
of ideal solid, 69 
of paramagnet, 94 
of two-level system, 93 

ideal Bose gas 
condensate of, 133-136 
critical occupancy of, 132 
critical temperature of, 132 
entropy in two-phase regime, 135 
entropy of, 128 
equations of state of, 129 
ground state of, 130 
heat capacity of, 131, 139 
low temperature limit of, 129 
multiplicity of, 127 
occupancy of, 126 
thermodynamic instability of, 133 
third law complicance of, 135 
two-phase regime of, 133 
ideal classical solid, 67-70 
relative entropy of, 70 
single particle partition function of, 69 
ideal Fermi gas, 110-118 
entropy of, 113 
equations of state of, 114 
extensivity of, 120 
ground state of, 117 
ground state pressure of, 118 
heat capacity of, 116, 120 
multiplicity of, 1 1 1-1 12 
occupancy of, 113 
of conduction electrons, 119 
of electrons in white dwarf stars, 119, 121 
of nucleons, 119 
third law compliance of, 116 
ideal gas, 66-67 

and occupation number method, 66 
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entropy of, 24 
equations of state of, 53, 57 
mixing, 73 
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relative entropy of, 25, 27, 53, 57, 67 
Sakur-Tetrode entropy of, 76 
independence of isolated subsystems, 41 
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energy equation of state, 109 
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multiplicity of, 108 

Jaynes, E. T., 156 
Joule expansion, 12-13, 45 
Joule, James, 19 
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laws of thermodynamics, 32 
Lorentz, H. A., 42 
Loschmidt, Johann, 4 
Loschmidt’s paradox, 4 

Mach, Ernst, 33 
macrostate, 34, 47 
improbable, 38 
maximum entropy 
principle of, 154, 156 
use in occupation number method, 60 
Maxwell, James Clerk, 47 
Maxwell’s demon, 47-48 
Maxwell-Boltzmann distribution, 64—65, 101 
microcanonical ensemble, 103, 154 
microstate, 34, 47 
classical, 45, 53 

missing information, 144, 145, 146, 159 
and statistical physics, 153-156 
in Einstein solid, 144 
multiplicity, 36, 51 
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of ideal Fermi gas, 1 1 1-112 
of ideal gas, 57 

of intrinsic semiconductor, 108 
of Shottky defect, 87 

negative temperature, 90-93 
experimental realization of, 92-93 
Nemst, Walther, 72 

occupancy 

critical value for ideal Bose gas, 132 
of ideal Bose gas, 126 
of ideal Fermi gas, 113 
occupation number method, 60-66, 

71, 110, 118 

applied to ideal classical solid, 67 
applied to ideal gas, 66 
maximizing entropy in, 62-63 
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paramagnet, 88-90 
Curie’s law applied to, 95 
energy equation of state, 90 
entropy of, 89 
heat capacity of, 94 
third law compliance of, 90 
particles 
classical, 35 
distinguishable, 35 
distinguishable versus indistinguish- 
able, 105 

fermions and bosons, 105 
partition function 

of a single particle, 63 
of ideal classical solid, 69 
of ideal gas, 66 

of quantized harmonic oscillator, 79 
of a system, 102 
Pauli principle, 106 
Pauli, Wolfgang, 106 
phase space, 54 

of quantized harmonic oscillator, 77 
phase transition 

of ideal Bose gas, 133 
phonons, 82 
and multplicity, 83 
photoelectric effect, 122 
photon, 122 

Planck, Max, 27, 39, 42, 49, 72, 75, 122 
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population inversion, 90 

quasistatic process, 4 

random variable, 99 

reversibility paradox, see Loschmidt’s paradox 
reversible work, 22 
room-temperature solid, 30 
rubber elasticity, 5 1 

Sakur-Tetrode entropy, 59, 76 
as limit of ideal Bose gas, 128 
as limit of ideal Fermi gas, 1 14 
second law of thermodynamics, 5-7 
and irreversible processes, 6 
and multiplicity, 43 
Carnot’s version, 19 
Clausius’s version, 5, 7, 18, 19 
entropic version, 12, 13, 18 
Kelvin’s version, 6, 7, 19 
semi-classical limit, 81, 114 
of Einstein solid, 93 
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of ideal quantum gases, 137 
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extensivity of, 146 
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of English text, 145, 148 
of multimedia files, 152 
Shannon, Claude, 140, 145 
Shottky defect, 87 

energy equation of state, 87 
entropy of, 88 
third law compliance of, 88 
spin, 106 

spin-statistics theorem, 106 
and Pauli principle, 106 
Stefan-Boltzmann law, 125 
Stigler’s law of eponymy, 99 
Stirling’s approximation, 38, 51, 56, 1 13 

temperature scales, 16-17 
absolute, 17 

thermodynamic instability of ideal 
Bose gas, 133 
thermodynamic process, 1 
adiabatic, 1 
irreversible, 3, 39 
isoenergetic, 1 
quasistatic, 1, 4 
reversible, 4, 7 

heating and cooling, 1 1 
thermodynamic system, 37 
thermodynamic temperature, see absolute 
temperature 

third law of thermodynamics, 9, 50, 85-86 
and blackbody radiation, 28 
and Boltzmann entropy, 85-86 
and Bose-Einstein condensate, 135 
and Gibbs entropy formula, 98 
and ideal Bose gas, 129 
and ideal Fermi gas, 116 
and ideal gas, 27 

and Planck’s convention, 27, 85, 86 
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nat, 142 
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